An Evaluation of the Promotional Processes in a

Large Texas Metropolitan Police Department

by

Stephen A. Bishopp

March 2010
 CHAPTER ONE

INTRODUCTION

Police officers make many decisions which can profoundly affect human rights

and often do so without direct supervision (Maher, 2003; Klinger, 2004; Lersch &

Mieczkowski, 2005). The officer’s behavior is often only constrained by the utility of

some form of social control (Black, 1976). Organizationally, police leaders should be

able to effectively oversee operations; enforce policies and motivate personnel and

provide an atmosphere of professional conduct (Stafford, 1989). Therefore,

understanding the process by which a person is chosen to be in a position of leadership

within the police organization becomes vital.

This current research was undertaken as an examination of the process by which a

police sergeant is promoted to the rank of lieutenant in a large metropolitan police

department; in particularly one utilizing an assessment center (AC) in determining

promotability. To study this issue, data were taken from a large Texas metropolitan

police department (LTMPD)1. In the LTMPD, the rank of lieutenant is an upper

management, command level rank. The data were examined in order to assess how a

person, through a dyadic promotional process (written examination and AC participation)

is able to arrive at this rank. This thesis provides participant/researcher insight into the

LTMPD’s promotional process in order to inform decisions concerning its continued use

for identifying successful police leaders.

The rationalization for analysis at this level lies in the fact that the rank of

lieutenant is the last or highest rank within the LTMPD which utilizes a test based

promotional process where numeric ordering provides a hierarchical list. Candidates who
take the lieutenant’s test and finish at or near the top of the final list are a revered cohort

and are the most qualified for promotion. The data will suggest the results of this dyadic

promotional process to be largely serendipitous.

Leaders are chosen to establish departmental goals and be able to lead officers to

successfully carry out those objectives (Scarborough et al, 1999; Berkshire Advisors, Inc,

2004; Klinger, 2004). Data from the lieutenant testing and promotional processes at

LTMPD during the past eight and a half year period were analyzed. The data consist of

three lieutenant promotional exams which have heretofore received neither critical review

nor transparent evaluation. A better understanding of what increases the odds for

promotion and, therefore, the behavior of the organization itself is provided. Because of

the organizational importance of the police leader, a discussion on police organization

and police behavior theories is necessary.

CURRENT THEORY

Police Organizational Theory

Lavigna (2008) described the importance of leadership to an organization as

“critical to organizational success” (p. 9). Leadership is particularly salient in policing

where preservation of laws and protection of human rights are of primary concern.

Shafer (2009) states plainly “the need for effective leadership in policing is quite evident”

(p. 238). This need is both external and internal to the police department in the way in

which it affects general operations. Externally, society should feel that the police

department has the legitimate right to use necessary power or authority over the citizenry

and that the police can do the job asked of them in a fair and proper manner. Internally, a

police leader must be able to direct work groups efficiently, ensure compliance to
policies, and generally see that the mission of the police department is carried out (also

see Jermier and Berkes, 1979). This is true for all levels of leadership, from front line

supervisors (such as patrol sergeants) to top departmental commanders and chiefs of

police. Shafer (2009) conducted a survey of 769 police commanders at the Federal

Bureau of Investigation’s National Academy. In his subsequent report, he noted that,

“ineffective leaders demoralize subordinates and co-workers, presented a poor image of

the agency within the community, and served as poor role models for future generations”

(pg. 248).

Police Behavior Theory

Moving beyond simple organizational only views, Klinger (2004) observed that

police officers act based on the interplay of two dynamics; the organizational structure of

the department they work for and the environment in which they are expected to perform.

This interplay has been largely ignored, as evidenced in the paucity of research literature

on this relationship (Klinger, 1997; 2004). Klinger addresses the matter head on and

develops a theory which has very important underpinnings for this current research study.

Briefly, he described organizational determinants of police behavior as properties such as

degree of professionalism and bureaucratization; department size, presence of specialized

units, and policy implementation. Environmental factors include judicial decisions (e.g.

rulings in evidentiary hearings or search warrant requirements), the political landscape of

those governing the police, and social ecology. These multi-faceted determinants are

important to understand when conducting an assessment of police behavior. An effective

police leader must be able to effectively implement policy while being responsive to the

needs of community in which he or she directs officer behavior.

The social ecology of the police is also important to understand in the context of
police leadership. Klinger (1997) notes the relationship between officers’ behavior and

“the level of social deviance in the patrol district in which they are situated” (pg. 300).

Klinger is saying that officers function within the context of the division or district in

which they work. For the purpose of the current research paper, Klinger’s patrol districts

are synonymous to the patrol divisions described in this current research in which daily

operations are largely reflective of the lieutenant’s leadership skills. In his article,

Klinger described the actions of officers as vigorous if they were likely to extend their

authority by making arrests, conducting investigations and so forth. The actions of

responding officers are viewed as lenient behavior when they are likely to do the opposite

and take no action. Hence, directing and monitoring the behavior of the officers and

setting their performance expectations are reflective of the leadership capabilities of the

lieutenants in command of the officers.

Within the LTMPD, lieutenants are the key in the supervision and management of

patrol practices within divisional boundaries. Larger cities are divided up into smaller

parcels in which police patrol (Wilson, 2003; Klinger, 2004). The LTMPD is broken up

into seven patrol divisions; each division is further divided by five to six sectors, and

each sector segregated further into beats. A lieutenant is generally responsible for the

entire division (having on the average between 200 and 300 subordinate sergeants,

corporals and patrol officers) much like the patrol division’s Deputy Chief. The Deputy

Chief has twenty-four hour responsibility for all divisional functions such as

administration, investigations, property maintenance, accountability, and so forth. The

lieutenant oversees these same patrol level functions during his or her eight-hour shift

(each division divided temporally by watches or shifts; day shift, evening shift and late
night shift and spatially by physical street boundaries, reporting areas, political

boundaries and, so forth). The lieutenant is responsible for the division’s officers’

service delivery, order maintenance, enforcement of state laws, and local ordinances

(Klinger, 2004). Additionally, lieutenants are responsible for subordinate officers’

adherence to departmental policy and discipline (Stafford, 1989). This is a tremendous

responsibility. Besides daily, routine operational oversight, lieutenants are expected to be

able to have boots on the ground in unexpected and extraordinary events which occur,

and direct line personnel during stressful events (e.g. hostage situations or riots). They

must be involved in planning and logistics during scheduled events to ensure effective

police presence at large gatherings (e.g. charity events or marathons). The police

lieutenant is also expected to direct the supervision of officers and accomplish established

goals during devastating natural disasters such as hurricanes, tornados or large fires.

The Management Efficiency Study of 2004

The number of lieutenants promoted in such a short period of time within this

urban police department demonstrates the need for an extensive review of promotional

processes. It must also be understood that nearly as many senior lieutenants would have

had to retire or otherwise vacate their rank to create those open positions. Berkshire

Advisors, Inc., (2004) provided a critical assessment of the management and efficiency of

LTMPD. They produced an aggregate level critical assessment of all operations and

processes within the police department which resulted in the publication of a

Management Efficiency Study (MES).

There are several key issues within the report which impact this research paper.

The deficiencies brought to light in the MES regarding promotion; training, competence,
and morale surrounding mid-level management (lieutenants) are disconcerting.

Examples of deficiencies in these areas include a near lack of past performance (positive

or negative) being reviewed and considered for promotion; lack of required

“supplemental supervisory training”, being out of touch with line personnel, and micro-

managing. Also, the MES discussed the potential for mass promotions in the mid and

upper management level ranks. Regarding this point, it stated, “The department is facing

a significant leadership vacuum over the next five years” (Berkshire Advisors, Inc., 2004;

Chapter II, p. 12). Two of the three tests occurred after 2004; the premonition of a

leadership vacuum appears to have come true.

The MES was initiated in order to evaluate the effectiveness of many areas with

the LTMPD. It identified policies and practices which needed improvement, and

suggested ways in which to make those improvements. Of particular interest to the

current research is the promotional schema of the LTMPD in light of the report because

the promotional process has not changed since the release of the MES.
 CHAPTER TWO

CURRENT RESEARCH

Although it gives rise to a number of important recommendations for change

throughout LTMPD, the MES is not the sole motivation for the current research paper.

There is a paucity of academic research specifically reviewing the police management

promotion predictors or the after effects. Much of the prior research focused on methods

of identifying officers worthy of promotion via AC (Ross, 1980; Feltham, 1988; Lowry,

1997 Brown, 2000; Ho, 2005; Lance, 2008; Rupp et al., 2008) promotions via

demographic or age related variables (Prewett-Livingston et al., 1996; Adams 2002;

O’Brien, 2003; Ho, 2005; Krajewski, H. and Rothstein, M., 2007; White, 2008; Archbold

& Schulz, 2008; Archbold, C., & Hassell, K., 2009) or other motivations for seeking (or

not seeking) to advance up the chain of command due to multi-cultural, financial, or

familial reasons (Whetstone, 2001).

The vast majority of research was conducted externally which means by an

observer or researcher outside of the process. None were done by someone who was

directly involved in and affected by the promotional process. The researcher on this

project was first an active participant in the three promotional processes, and then later an

academic researcher examining the process in which he had participated. It is from this

methodological approach that current research will shed light on this promotional

process. Furthermore, it is the intent of this project to contribute quantitatively to the

extant literature on police promotions and provide a benchmark by which further

academic analysis of promotional processes can be conducted. The current research also

suggests a theory for police organizational success. In order to obtain effective police
leaders, an organization must promote those with the ability to efficaciously provide a

positive work environment for subordinate police officers as well as build police

legitimacy within the community.

Based on the importance of leadership and considering the supervisory finesse

needed to successfully function organizationally and environmentally, one would expect

promotions to be based on merit or performance. The LTMPD’s use of the lieutenant’s

assessment center is used to gauge a candidate’s managerial merit and predict one’s

ability to successfully perform as a lieutenant. Some may argue that the AC process is

able to effectively evaluate candidates and predict future success and therefore justify its

high financial cost2 (Krause et al., 2006). Those who are able to display leadership skill in

the AC must surely have merit as a manager. Many municipalities appear to have bought

into this concept of the AC’s ability to assess leadership qualities. A survey found 62%

of cities with populations over 50,000 used the assessment center for police and fire

promotions (Lowry, 1997). However, despite the AC’s abundant use the current

literature is far from congruent on the issue of AC validity3.

In light of the extant literature this research hypothesized that many variables are

unconnected to merit but are connected to promotion. This paper reviews the literature

on predictors of promotion. Next, there will be a discussion of the importance of

effective police leadership by describing what the dangers of poor leadership can be.

Following that, methodology and results are presented. Finally, this paper wraps up by

discussing the findings and the variables which either influences or has no influence on

being promoted to lieutenant within the LTMPD.

VALIDITY AND ASSESSMENT CENTERS

The LTMPD Assessment Center

There are a number of variables being analyzed in this thesis as a predictor of

promotion. Chief among these variables is the AC score earned by the promotional

candidate. The reason for choosing this as the most important independent variable is

simply because of the formula used by the City of Dallas to determine an overall score.

That formula, written score (40%) + assessment center score (60%) = 100% of the final

score. Hence, it is reasonable to assume that the greatest predictor variable will be the

assessment score. The following section discusses and reviews the research on

assessment centers and the validity of those measures within the context of the law

enforcement promotional process.

This research utilizes Ross’ definition (1980) of the term ‘assessment center:’

“The term ‘assessment center’ describes the simultaneous assessment of several

individuals conducted by a group of trained evaluators using a variety of exercises
and providing evaluation of a number of abilities.” (p. 89)

Simply stated, this means a number of people (assessors) evaluate candidates in

several scenarios (in this case study of a LTMPD it is three scenarios) to

determine or judge the candidates possibility of success as a police lieutenant.

During each assessment session, the candidate is provided a timed problem or

situation and is graded on his/her ability to perform as a lieutenant in the context

of several specified dimensions (Ross, 1980; Feltham, 1988; Rockhill & Halton,

Inc, 2007). As an example, the lieutenant’s test for the LTMPD measures six

dimensions; written communication, oral communication, supervisory

skills/administrative skills, problem analysis, interpersonal relationships, and

leadership. What should also be understood is what the assessors are not

supposed to be grading. These dimensions are; job knowledge, understanding of

the job effectiveness under stress, and attention to detail. The unevaluated

dimensions are purported to be embedded in the exercise but they are not graded

(Rockhill & Halton, Inc, 2007).

The AC for the LTMPD takes place over three consecutive days in which

a tactical exercise, oral presentation exercise, and an in-basket exercise3 are

presented; one exercise per day. The oral presentation and tactical exercise are

structured in the same way; thirty minutes to review written material and take

notes, a ten minute break, and then ten minutes of interview by two or three

assessors. For the in-basket exercise the candidate’s exercise time is extended;

one hour to review material and take notes, five minutes to put the reviewed

material in a stack with the highest priority item first. The candidate then gets a

thirty minute break after which twenty minutes is allotted for the assessors to

interview the candidate and ask questions concerning the in-basket. What this

essentially represents is a period of time no longer than forty minutes spread out

over three days by which the assessors decide who has the skill to be a lieutenant.

Another important aspect of the AC should be brought to light. During each

allotted time period; ten minutes on day one, ten minutes on day two, and twenty

minutes on day three there are a different group of assessors during each exercise.

That means no set of assessors sees the same candidate over the entire forty

minutes. Essentially, leadership/managerial skill decisions are made in

increments of ten minutes, ten minutes, and twenty minutes.

With the AC explicated, the discussion moves necessarily on to the

validity of the ACs and the information they provide via this portion of the
promotional process. The validity of police selection processes has been

challenged many times in the past concerning their predictive value for future

police performance (Ho, 2005). The next section reviews at some length the

literature on AC validity.

The Types of Validity Related to Assessment Center Measurement

LTMPD trusts that AC’s are valid for police promotions. More

specifically, they rely on the strength of the ACs capability to accurately measure

and determine the level of leadership skills in tested individuals, thereby

predicting a high likelihood of success at the rank of Lieutenant. Additionally,

this is the highest rank currently attainable on the LTMPD for which one

competes for promotion by means of testing (the next level is the deputy chief

rank and is a command level position appointed solely at the discretion of the

Chief of Police). Analyzing the testing process to determine how one becomes a

lieutenant is more crucial for this position because this is the rank by which the

police department’s chief level officers are selected. Because of the assumed

strength of the relationship between AC scores and promotion, it becomes

imperative to review the research on the validity of assessment centers.

Ross (1980) conducted a criterion related validity analysis of AC use in

law enforcement promotional processes. Criterion validity refers to the ability of

the AC to rate variables such as judgment, oral expression, problem analysis and,

leadership (Ross, 1980; Feltham, 1988; Rockhill & Halton, Inc, 2007). This is

accomplished through “focus on the correlation of the test being validated with

some well-respected outside measure of the same objectives” (Brown, 2000; p. 8).
Ross’ (1980) analysis was of a southern California police agency which rated 49

police promotional candidates. The purpose of the study was to determine the

degree to which an AC was capable of predicting effective and ineffective

management behavior. The determination was made via a two year follow-up

using performance evaluations as a measurement guide.

This study revealed several key points. Ross (1980) found that the

Pearson’s correlation between job performance and AC ratings was .47, a

moderate to high level of correlation (Maxfield and Babbie, 1995). Ross then

compared this correlation to the correlation between written test scores and job

performance. Ross (1980) found that written tests were correlated with job

performance at a level no higher than .30 (also see Feltham, 1988). Secondly,

verbal skills, rather than job knowledge, organization, writing skills, etc., were

heavily relied on (Ross, 1980). She stated that “it appears that the assessors relied

mostly on evidence of interpersonal skills and communication skills to make their

final rating” (Ross, 1980, p. 94). The study demonstrates the value of rating the

AC process post-promotion in order to judge the predictability of the AC.

One other important concern discussed by Feltham (1988) was the

cognitive frame through which assessors rated promotional candidates. For

instance, the author found that assessors rated individuals similarly. However, the

assessors had differential definitions of the various job performance criterion used

to rate the candidate. Furthermore, he states, “another explanation could be that

factor analyses of performance ratings indicate more about the cognitive structure

of the raters than the behavior patterns of the ratees” (p. 139). This appears to
point to a realization that raters perceive the same job performance differently

because of their own perceptions. Therefore, one’s overall rating may have much

more to do with the raters than with how the promotional candidate actually

functions during the AC.

Content validity describes the ability of the AC exercise to evaluate the

objective it is meant to gauge. In other words, if the candidate is being assessed

on his or her ability to display leadership and interpersonal skills, does the subject

matter contained within the exercise actually target those dimensions? Content

validity is based on the particular tasked being performed. Lowry (1997)

purported this approach to AC testing as a better way to actually gauge a

candidate’s ability at getting a specific job done. For example, when rating a

candidate’s organizational skills, a scenario may present several issues to the

candidate; notify personnel under their command of meeting, discuss a possible

agenda with peers, pick the place for the meeting and, brief the chain of command

(i.e., the deputy chief under whom the lieutenant serves) of the content and

outcome of the meeting. A check list of typical actions could be used to indicate

when each action or task is accomplished (Lowry, 1997). He goes on to state,

“The task-specific assessment center is concerned primarily with

evaluating the subject’s performance in tasks (work samples) related to the
job. Therefore, the desired outcome from such a job analysis should be
the tasked performed on the job, rank ordered based on the importance,
frequency and need to perform on entry to the position” (Lowry, 1997, p.
55).

Lance (2008) also argues for the advancement of content validity rated

AC’s. He posits that when assessing dimensions (as occurs with construct and

criterion validity measures) such as leadership or communications skills, the

candidate is often assessed based on how well he or she performs within the

context of the exercise setting rather than the actual portrayal of the dimension.

In his investigation of assessment center validity Lance (2008) observed “factor

analysis almost always support robust exercise factors and not dimension factor”

(p. 87). Rather than capture the behavioral dimension for which a particular

exercise is designed to measure, the exercise itself becomes the evaluated task.

This is an important finding when taking into account the LTMPD AC

process. In each exercise scenario the candidate is put into a role and presented a

problem. For example, he or she may be told to act as though they are at a city

council meeting and addressing council members. The candidate is given the

problem to address and a maximum of ten minutes (Rockhill & Halton, Inc, 2007)

to “act” out the scenario. It can be easily deduced that a person who is better at

acting as a manager (lieutenant) may fair better than the person who is, in fact, the

better manager but with poor acting skills. Furthermore, Lance (2008) makes the

argument that the AC does not measure those dimensions for which they were

designed. He found that assessment centers “evaluate candidate behavior with

respect to potentially content valid simulated performance situations” (p. 93), and

not psychometric dimensions.

Rupp et al. (2008) dispute Lance’s finding and observe that AC’s do have

the ability to measure the dimensions of the candidates. They go one to argue that

the dimensions are the sum total of numerous abilities needed to affect good job

performance. They state that, “knowledge, skills, abilities, and so on that are

identified as (a) job relevant (b) most appropriately accessed via behavioral
methods.” Furthermore, one could expect the role of the manager to be

multidimensional when performing a specific task. Assigning personnel certain

days off while keeping in mind each person’s unique life situation vis-a-vis

departmental needs and goals. In an exercise of this type the candidate would be

assessed within several dimensions; leadership, interpersonal skills,

communication, and so on. Rupp et al. (2008) state that it is peculiar to measure

one task at a time when one task at a time is not how the ‘real world’ works.

Construct validity concerns the utility of an AC in transforming a

dimension or construct into a quantitative and measurable variable. Brown (2000)

describes construct validity as “experimental demonstrations that a test is

measuring the construct it claims to be measuring” (p. 8). The question here is

whether or not an exercise aimed at measuring a construct is able to actually judge

and measure the candidate’s ability. The AC used by the LTMPD rates the

following dimensions or constructs; written communication, oral communication,

supervisory skills/administrative abilities, problem solving, interpersonal skills,

and leadership. (Rockhill & Halton, Inc, 2007). Brown (2000) suggests that there

is no single way to measure the construct validity of a test. Furthermore, construct

validity measures are likely to have problems of ambiguity caused by attempting

to define the construct by which a candidate will be evaluated. Lowry (1997)

observed that there are over 8,600 definitions for the dimension of leadership. He

also suggests using task-specific measures rather than trying to quantify complex

constructs.

Importance of Validity
 According to Kroecker (2000) assessment centers are a way in which

police departments can more effectively measure those abilities not captured

through written tests or performance plans (job evaluations)4. Kroecker indicates

the AC is the preferred method and should be given greater weight in predicting

of future behavior over performance plans, written scores, and even extensive

service. The reason is that a promotional candidate may have superior strengths

in all of these areas but lack the ability to think critically or manage personnel.

Even though a heavier weight being given to the AC is suggested, Kroecker

(2000) advances the opinion that AC score should not be the stand-alone decider

for promotion. Other indicators of performance (i.e., interviews, minimum

number in current job, or overall tenure) should be considered as equal pieces of

the promotional pie.

Made clear in the article on AC benefit, Kroecker (2000) explicated the

insurmountable importance of validity. It is absolutely essential for an AC to

provide valid results in order for it to predict leadership qualities. Feltham (1988)

offered a caveat when evaluating assessment center validity. Psychometric

problems (the measurement of personality traits for example) can create undesired

results. He posits that the halo effect, for instance, is the most intractable.

Feltham (1988) further suggests incorporating a more comprehensive assessment

of validity from such sources as subordinates and peers to allow for a more

complete picture of promotability.

DEMOGRAPHIC PREDICTORS

Age
 The influence of age has been analyzed in recent research (Adams, 2002;

Krajewski, & Rothstein, 2007; White, 2008). Of particular interest in the articles

is how the age differential effects decisions to promote within a police

department. One may expect minimal promotional effect in an organization

operating within the constraints of a paramilitary rank structure. The ADEA (Age

Discrimination in Employment Act) places federal restrictions on employment

status related to age. The law also affects the use of age as a variable utilized as a

decision factor for promotion. That is not say age has zero effect. For instance,

one must be at least 19 ½ years of age6 in order to be hired; serve a specific period

of time at each rank before being considered for promotion to the next level, and

so forth. Therefore, while age necessarily has an effect on time-in-grade (amount

of time spent at a rank) and time-in-service (aggregate seniority on the

department), chronological age should have no effect on who does and does not

get promoted. According to the analysis of the date from the 202 promotional

candidates of this thesis, this does not appear to be the case.

The discussion in recent research indicates age may very well play a role

in who is deemed worthy of promotion. This effect can be negatively correlated

to promotion (Adams, 2002), or have a positive effect on AC scores for older

individuals (Krajewski, & Rothstein, 2007). Krawjewski and Rothstein (2007)

tested the effect of age on AC performance and found that older individuals may

enjoy an advantage. An examination of 371 police managers revealed that while

the effects where moderate; there was support for use of age as a predictor for

promotability of police managers They observe, “the current study provides some
support for this suggestion by showing that certain traits are more highly related

to AC scores when age is taken into account” (pg. 30). Further noted by

Krawjewski and Rothstein (2007) was the suggestion that AC assessors may be

using stereotypical processes when faced with large numbers of candidates. They

posit a halo effect by the assessors by which older candidates were predetermined

as more capable “in order to facilitate the complex decision making and rating

tasks required of them” (pg. 29).

Interestingly, an opposite argument was offered by Krawjewski and

Rothstein (2007) with stirs the debate on the mitigation the age factor. Possibly,

older candidates may be stereotypically viewed as being less energetic or slower

to learn; hence the older individual is less desirable for consideration of

promotion. However, should a candidate be able to cross the age stereotype, their

AC score would be considerably higher.

The argument of the age factor being utilized and inversely correlating to

promotion was analyzed in an article by Adams (2002). While not within the

context of policing, the research clearly points out the age effect during the

promotional decision making process. Adams (2002) suggests older workers are

passed over for promotion but economic reasons drive those decisions. Career

events such as retirement packages for older workers and the creation of job

attachment through promotion of younger employees may be driving factors in

career advancement decision choices by company leaders.

The studies by Krawjewski and Rothstein (2007), and Adams (2002),

make viable cases concerning age mediating the decision to promote. Both are
important to this current research because they demonstrate age may be a

controllable factor for analyzing a person’s promotion within an organization. It

may be an economically based option for a company or the employee, the effect

of AC assessors’ preference or the candidate’s superior demonstration of

managerial traits; either scenario demonstrates that age cannot be simply

dismissed as a predictor of promotion.

Gender

There was paucity of research available specifically addressing the subject

of female promotion rates within a police department. Participation in testing and

eventually getting promoted is a self-selected process and is necessarily affected

by the decision to not seek promotion. While it has been suggested that males fair

better during academy training (White, 2008), the issue of promotions and the

gender effect is largely unexplored (Archbold & Schulz, 2008; Archbold &

Hassell, 2009).

The gender question on promotional outcomes was examined within the

context of making a decision not to promote. Tokenism was seen as one such

reason. Archbold and Schulz (2008) suggested that tokenism in the police

workplace is more complex and could not be seen as a simple numbers game. A

female who is seen as a token promotion (or view themselves as a token) “are

faced with high visibility, isolation, problems assimilating into the group, and

performance pressures that stem from a combination of these factors” (Archbold

& Schulz, 2008; p. 52). Females may choose not to promote even if they believe

they are competent and capable of fulfilling the role if promoted, they decide not
to risk being seen as a person promoted based solely on their gender.

In the similar vein of opting out of the promotional process, Archbold and

Hassell (2009) found that females’ reasons for deciding not to promote were very

closely related to a man’s reasons. In a list of top five reasons for women and

men declining to participate in the promotional process, the lone difference was

administrative bias. The bias is related to tokenism and that females may decide

against promotion because they are being pushed by their bosses to participate

and promote. Therefore, it is felt as though they are being singled out and

eventually promoted because of gender, not ability.

Archbold and Hassell (2009), however, point out that it is the “marriage

tax” that may be inhibiting female officers from seeking promotion. They must

consider child care, family schedules, and pay when deciding to advance their

rank. Organizationally, females are faced with (or more aware of) anti-nepotism

rules. These policies “restrict the promotional opportunities for women who are

married to male officers” (p. 70). While this certainly may be the case, especially

in smaller departments, a large metropolitan area consisting of more than 3,000

officers (e.g. the LTMPD) would certainly have a greater number of rank specific

career opportunities; thereby, nepotism would be of less concern. The gender

variable, however, cannot be overlooked.

Race

Affirmative action promotions and race quotas have been analyzed in

several recent articles (Prewett-Livingston et al., 1996; O’Brien, 2003; Ho, 2005).

Although not specifically addressing promotions to managerial ranks, this

research is insightful concerning hiring policies that can be directly related to

departmental promotional polices. For instance, the predictability of race on

determining employment would naturally be transferable to policies directing

promotional processes. Therefore, affirmative action requirements and quotas

used to increase employment among certain demographically situated individuals

should be assumed to increase minority representation within a particular rank.

O’Brien (2003) found that this may not always be the case. He analyzed the

initial employment of minority group members regressed on variables found

within the population (number of minorities in population; also see Ho, 2005),

and governmental structures (race of mayor and race of city council

representatives). Not unexpectedly, he found populations with a greater number

of minorities “had the most consistent, positive but modest, effect on minority

employment” (p. 194).

Interestingly, O’Brien (2003) found that policies utilizing affirmative

action without using quota goals were not found to increase the representation of

any particular minority group. However, used in conjunction, Black and Hispanic

employment increased. In addition, higher pay actually accounted for decreased

minority employment (O’Brien, 2003). In contrast, Ho (2005) found “race-based

favoritism toward ethnic minority applicants might not exist in the police

departments recruitment decisions” (p. 470). But, affirmative action was found to

increase metropolitan minority police officer ranks. Ho (2005) also found

psychological recommendations were not favorable to minority applicants.

Recruiters did not place a high importance on those recommendations so the

impact on minority hiring was not significant. This possibly reflects the

importance of affirmative action goals being placed over psychological

evaluations. Either way, in light of these current analyses, a possible race based

paradigm among those designing and executing hiring examinations exists.

Therefore, it can be naturally assumed that race plays a role in the promotability

of a particular person based on demographics.

Further muddying the waters of the race component and promotions,

Prewett-Livingston et al. (1996) provide evidence that race and final AC scores

are related. They contend that AC assessor race influences final ratings.

Particularly, they investigate possible similarities in race between assessor and

candidate influence final score outcomes. They posit “members of groups (e.g., a

racial group) will tend to evaluate members of their group more positively than

members of other groups” (p. 179).

This study specifically addressed a police manager AC and the final scores

assigned to candidates being a function of assessor panel race distribution.

Prewett-Livingston et al. (1996) analyzed the a sergeant’s AC (N=135) and offer

the summation,

“In summary, despite its limitations, the present field study offers
evidence that rated performance of Black candidates and White candidates
in a selection panel interview was a function of the Black-White
composition of the interview panels and the race of the panel
interviewers.” (p. 185).

While the limitations in this study included being unable to completely control the

racial makeup of the panel interviews (assessors) and accuracy of the panel scores

being unavailable, this study certainly points a statistical finger in the direction of
“race matters” for promotional considerations. Hence; race must be included in

any examination of police promotional processes.

Education

Since August Vollmer was Chief of Police in Berkley, CA, at the beginning of the

twentieth century, higher education has been proposed and debated as a requirement of

employment as a police officer. Education is seen as something that should be ongoing

(Travis, 1995); essential to effective community policing (Breci, 1994); or as having little

or no influence on successful police training (White, 2008). The federal government

weighed in on the issue of higher education for police officers through The President’s

Commission on Law Enforcement and the Administration (1967) which made the

following observation and recommendation,

“A policeman today is poorly equipped for his job if he does not

understand the legal issues involved in his everyday work, the nature of
the social problems he constantly encounters, the psychology of those
people whose attitudes toward the law differ from his. Such understanding
is not easy to acquire without the kind of broad general knowledge that
higher education imparts, and without such understanding a policeman's
response to many of the situations he meets is likely to be impulsive or
doctrinaire. Police candidates must be sought in the colleges and
especially among liberal arts and social science students” (p. 107)

The question of the level of education obtained by a police officer and its effect

on the officer’s performance and/or suitability for a policing career continues to be

discussed extensively in more recent literature. (Bayley, 1994; Breci, 1994; Travis,

1995; Mastrofski, 2006). More precisely, the quandary has been directed at how much

education is needed to have a positive influence on the officer. Mastrofski (2006) wrote

that there was no evidence that there was an impact on officer decision making (an

integral dimension for successful policing). Mastrofski asked several questions which
helped explicate the gap in the research evaluating the positive or negative effect of

higher education on police officers. Two questions are particularly poignant: (1) “What

skills and habits, if any, does college develop?” and (2) “Do some police assignments

benefit when college-educated officers perform them but others do not?” (p. 7).

Neither of these questions will be specifically answered by the current research.

The use of educational attainment as a predictor of police promotions, however, may

shed some light on the extent to which education plays a part in the promotional process.

Certainly, skill and the ability to perform a particular duty (Breci, 1994), such as one

required of a lieutenant, are important, it would be beneficial to know to what extent

higher education plays in one’s promotion to lieutenant. This debate takes us a step

closer to understanding the importance and connection of obtaining a higher education

and its influence on the promotion of a police lieutenant within the observation group.

The next logical step would be to gauge the effectiveness of a college education in

predicting the promotion of police commanders. Or, just as importantly, the

ineffectiveness.

Higher education is thought by some to be the future of law enforcement (Bayley,

1994; Breci, 1994). Such reviews speculate that the “complexities of policing would

require higher levels of education” (Breci, 1994, p. 1). Bayley (1994) recognized that a

higher education is an important variable for senior police promotions on numerous

police departments throughout the world. He observed the average educational level of a

police officer rose from a high school diploma in 1969 to at least a two-year college

degree by 1990.

On the other hand, White (2008) analyzed police officers’ performance by way of
measuring their performance in the academy. The premise is that officers who

demonstrate superior performance in the police academy will most likely be better

officers on the street. That being said, White (2008) attempts to predict officers’

academy performance by controlling for several variables, college education being one of

those. There can be little argument against a police officer gaining more education.

Unquestionably an academically well rounded officer could be expected to make better

and more objective decisions. Through a multivariate analysis, he finds that the

possession of a college education is not statistically significant in the prediction of

successful academy performance.

The literature does not quantify the extent to which education is able to actually

predict a skillful leader. There is a questionable gain by obtaining a college education

when the effect may not last a significant amount of time (Mastrofski, 2006). However, a

criteria or qualification which is important for initially hiring a person can be inferred as

being even more important when one is promoted to a managerial rank. Obtaining a

college education (or at least some college credit) is used by some police departments as

a hiring requirement and seen as facet of future policing. However, it is hotly debated as

to the importance or influence education should have in hiring or promotion of police

officers. More study is needed to properly quantify the effect education has on successful

police leadership. This research attempts to take a step in that direction, albeit small, by

controlling for education and operationalizing it as a dummy variable. This analysis will

allow for a measure of the effect of education on the promotional process without being

confounded by data showing number of hours of higher education one may have.

Past Behavior, Tenure, and the Number of Times Tested for Promotion
 Danner et al (2008) studied the effect of habits as predictors of future behavior.

They found that habit attenuated the effect of intention-driven behavior. The incidences

of commendations (good behavior), and sustained complaints (bad behavior) indicates

frequency of past events; therefore, it describes the candidates past behavior. The

number of times tested indicates a past behavior that is intentional but not necessarily

habitually performed. This research tests commendations and sustained complaints, and

the number of times tested to analyze their effect on the promotional process. Further,

the promotional candidate’s tenure is expected to have a direct correlation to times tested

and the number of commendations and sustained complaints due to temporal conditions.

Those who have been on the LTMPD longer had more opportunity to take numerous

tests. Their longevity also afforded them more opportunity to receive commendations or

sustained complaints. Additionally, there is very little quantitative research on the effect

of tenure as a predictor of future behavior. Therefore, tenure is not used utilized as an

independent variable but is provided in Figure 1 for illustrative purposes.

DEMONSTRATING THE NEED FOR EFFECTIVE POLICE LEADERSHIP

Hays et al. (2007) accurately and succinctly state, “Strong leadership is essential

for effective policing” (p. 3). Strong leadership, however, creates several connotations in

what it takes to be a strong leader. Rowe (2006) and Densten (2003) note the importance

of portraying leadership qualities to encourage “followership” within subordinates.

Among some of the issues addressed in assessing effective (or strong) leadership are such

variables as time in grade (Rowe, 2006), education (Hays et al., 2007), and behavior

modification (Trautman, 2002; Hubert et al., 2007), Rowe (2006) explicated a common

theme in via an ethnographic study of lower rank officers’ perceptions of senior

management. He found that on average the street level officer saw senior management as

detached from day to day operations or “real police work.” This leads to a loss of

legitimacy in decision making skills of supervising officers and creates a milieu of

subordinate officers who have little faith in their commanders. Rowe (2006) further

suggests a time in grade requirement ought to be considered. Officers should be expected

to stay at a certain rank for a substantial period of time in order to gain experience before

moving up to the next level. He found, “That the rapid ascent of some officers up the

rank scale meant that they did not have time to develop operational experience was often

noted as problematic by PCs [police constable or officer]” (p. 765).

Beyond the subjective and sometimes narrow view taken by lower ranking police

officers there lie some real, objective problems. When there is a failure of proper

supervision or appropriate oversight by senior police managers there can be

departmentally damaging misconduct. This misconduct can range from overlooking

obvious ethical problems, improper or inefficient training program, refusing to be held

accountable for decisions and so on (Trautman, 2002) which may lead to systemic morale

issues and ethical problems with personnel.

Illuminating the damaging effect of poor supervision, Pogarsky and Piquero (2004)

applied deterrence theory to the study of police conduct. They found perceived sanctions

which come certainly and quickly lowered instances of police misconduct. In other

words, if an officer knows for sure that his or her misconduct will be found out and

quickly punished, they will most likely be deterred from it. A common theme found in

the literature is a police manager who is involved with his own misconduct is far less

likely to observe or punish the misconduct of others.

 Promoting a person who has demonstrated past performance problems or created

disciplinary issues through their own misconduct is not conducive to good leadership.

While there some argument as to how much or to what extent past behavior predicts

future performance (Ouellette & Wood, 1998; Ajzen, 2002), there can be little doubt that

one does not magically gain police knowledge, morals, or virtues simply by being

promoted. Strong leaders possess these tools before they come to the table.

Huberts et al. (2007) identified three leadership qualities and their effects on

police integrity violations. They found that role modeling, strictness, and openness of the

manager have a significant effect on most of the twenty integrity violations analyzed in

this study. The police leader must display those characteristics he or she wishes to see in

officers under their command (Densten, 2003) and lead by example (Engel, 2003). Engel

(2003) identified four styles of supervision, each having a positive and negative

characteristic. The commonalities in the positives were such concepts as the ability to

form relationships with subordinates and being supportive of decisions made by lower

ranking officers. In addition, the most productive style of supervision was one in which

the supervisor was actively engaged. Engel (2003) describes this style of supervisor as,

“Active supervisors embrace a philosophy of leading by example. Their

goal is to be heavily involved in the field alongside subordinates while
controlling patrol officer behavior, thus performing the dual function of
street officer and supervisor” (p. 6).
HYPOTHESES

This evaluation of the promotional process is done in order to answer the basic

question: what does it take to get promoted? Organizational theory posits that the best

and the brightest should rise to the leadership roles. This ensures the organization’s

success and is true for any organization, private or public. It is a theory not specific to

police organizations. Klinger’s (1997) integrated police behavior theory gives a very
insightful glance at police behavior at the individual level as well as at the organizational

level. Considering these two theories together, the importance of having a means by

which to choose effective police leaders can easily be seen. Theoretically, the police

leader has to be able to successfully function in the police milieu while simultaneously

juggling the environmental factors external to the police organization. For this analysis

the lieutenant rank is purposefully targeted because it is the highest position for which

one may test. Most importantly, though, it is this rank where the success of the

organization relies on how well the organizational rubber meets the environmental road.

Following previous research (i.e., Ross, 1980), this study examined the following

hypotheses. The first set of hypotheses tested in this current research attempts to

examine what impacts the AC outcome (See Table 1 Below for Summary). AC’s

assessors rate promotional candidates on acting skills (Lance, 2008), job specific

behaviors (Lowry, 1997), as well as interpersonal skills and communication skills (Ross,

1980).

What is known about the current data is that one must score high enough on the

written test to qualify for the assessment center. Candidates are given a 100 point written

test and a passing score is assessed; therefore, the written score coefficient is expected to

be in the positive direction. All candidates who score at or above the passing score are

invited to continue with the second step of the promotional process, the AC. It is

hypothesized that the written score will have a positive influence on the AC score. All

other variables are not factored into the final score and should have no effect on the AC

score. However, due to the extant literature, final AC score will likely be influenced by

those variables.
 Education plays no direct role in one’s qualifications and eligibility to be

promoted to Lieutenant. However, a certain level of education must be obtained to be

hired on the LTMPD so education is expected to impact promotability in a positive

direction. Therefore, it is hypothesized that education will have a positive effect on ones

promotion.

The percentage of females on the LTMPD remains remarkably similar to state and

national percentages of representation on the police force as an entire unit. All police

departments are expected to maintain representative diversity in their hiring and

promotion practices (EEOC, 2010). Thus, this research expects that being a female

sergeant will positively correlate with promotion. However, much of the extant literature

concerns candidate demographics such as age and minority status. For instance, age

related to assessor perceptions of promotability can both negatively and positively affects

final scores (Adams, 2002; Krajewski and Rothstein, 2007). It is expected that age will

have an effect; however, it is unclear from the conflicts in the extant literature whether

the impact will be in a positive or negative direction. Additionally, race is expected to

affect the promotion decisions. Prewett-Livingston et al (1996) found that race

(minority) also has a discernable effect on how raters score candidates during AC’s (also

see O’Brien, 2003; Ho, 2005).

Sustained complaints and Commendations are used as proxy measures of the

candidate’s past behavior. Behavioral dimensions (e.g., leadership, communication,

organizational, and interpersonal skills) are assessed during the AC and some

measurement of one’s past is appropriate. While it is arguable that past behavior is a

good predictor of future behavior (Ouillette and Wood, 1998; Ajzen, 2002), the proxies
utilized here are the written documentation of positive and negative behavior. As such, it

is expected that commendations will have a positive effect on AC scores while sustained

complaints should have the opposite effect. The number of times tested is regressed on

AC score because literature suggests that one is capable of learning how to do the

assessment center (Kroecker, 2000; Rockhill and Halton, Inc, 2007). This learning takes

place via feedback from prior AC as well as training classes prior to the AC. Each time a

person participates in the AC process he/ she learns how to better himself/herself and

prepare for the next promotional process in which they participate. The number of times

one takes test is expected to improve promotional odds.

Table 1: Summary of Expected Direction of Influence on Assessment Center Scores

Variable Direction of Expected Relationship

Written Score +
Education +
Female +
Minority +
Age in Years -
Commendations +
Sustained Complaints -
Times Tested +

The second set of hypotheses concerns the weight given to the AC score in the

final score, this research hypothesizes that attaining a higher AC score will better predict

who is to be promoted. Furthermore, written score should also report as significant and

in the positive direction. All other variables shown in Table 2 are expected to have

similar predictability in strength and direction on promotions as they had on high

assessment center score. Because the AC score is weighted so heavily in the final score

outcome, anything affecting the AC score should similarly predict promotion. That being
said, it is believed that the AC score, following prior research, will be the largest

contributing factor in the promotion decisions at LMTPD.

Table 2: Summary of Expected Direction of Influence on Promotion

Variable Direction of Expected Relationship

AC Score +
Written Exam Score +
Education +
Female +
Minority +
Age in Years -
Commendations +
Sustained Complaints -
Times Tested +
 CHAPTER THREE

DATA AND METHODS

DATA

The data were collected from a series of open record requests for information.

The information collected was all deemed to be theoretically relevant to the study at

hand. It should be noted that the dataset is a population rather than a sample. That is, the

individuals that are represented in the data include every person that participated in at

least one (or as many as all three) of the promotional process examinations. Data

included information about the sergeants’ demographics, their work records, and their

testing records. (See Table 3 below for a synopsis of the descriptive statistics of all

variables.)

Table 3: Descriptive Statistics of Lieutenant Promotional Population Data

Observatio
Variable Name ns Mean Std. Dev. Minimum Maximum
Promoted 202 0.3465 0.4770 0.00 1.00
High Written Score 202 73.3069 8.0047 57.00 97.00
High AC Score 202 7.3037 0.9446 4.52 9.36
Education 202 0.7277 0.4462 0.00 1.00
Female 202 0.1733 0.3794 0.00 1.00
Minority 202 0.4703 0.5004 0.00 1.00
Commendations 202 55.2079 24.6132 12.00 151.00
Sustained Complaints 202 0.8119 1.4573 0.00 7.00
Times Tested 202 1.3218 0.5467 1.00 3.00
Age of Sergeant 202 41.3502 5.3048 30.20 58.94
Source: Open Records request via Public information Act (Texas Open Records
Act), Chapter 552, Texas Government Code (2009).
The present study is an amalgamation of testing procedures, formulas, and

individual level data from the entire population of 202 police sergeants who tested for the

rank of lieutenant. The data presented in this research is collected from three temporally

separated lists of promotional prospects. Promotional processes represented in this study

are dated by the date of the written test. Those dates are November 1, 2001; October, 20,

2005, and October 30, 2007 tests and naturally provide excellent quantitative data. The

written test date is chosen as the beginning of the promotional testing course because

after the process is complete and a list of candidates provided, actual promotion dates

vary greatly. For instance, a list of the numerically ranked candidates is given to the

Chief of Police. He may promote the first 2 one month after the final list, the next 5 ten

months after, etc. The actual promotion dates may vary greatly even within the same list

of candidates. However, the written test date remains the same for everyone on the list.

Dependent Variables

Two statistical models were completed during the analyses, and thus this research

had two dependent variables. The first dependent variable is the AC Score, which is

measured as the value assigned to the promotional candidate by the Assessment Center.

The second dependent variable was promotion. This was measured simply as whether or

not the Sergeant was promoted to Lieutenant after completion of the testing process.

Promoted is coded simply as a dichotomous variable (0=Not Promoted and 1=Promoted).

Independent Variables

Several variables were used to examine the various influences on both of the final

outcome measures of this research study. First, the written test is a test where the

individual can score up to 100 point test. The data from this population shows that actual

scores ranged from 57 to 97 and had a mean score of 73.31. In order to qualify for the

assessment center, the candidate must score higher than a set minimum score. The

qualifying minimum scores were different for each test and were not announced until all

scores have been tallied; only those with the higher scores are allowed to proceed in the
testing process7. Assessment center scores were weighted and then transformed into

scores that could range from zero to ten. The population showed a mean score of 7.30 and

a range of 4.52 to 9.36. All three AC scores were transformed to be equivalent measure

for the performance of the analyses in this research paper (See Appendix 1 for the

formulas used to make this transformation).

Demographic variables were also included about each individual officer.

Education was coded as a dummy variable whether a candidate completed some college

hours (1) or had not taken any college credit hours (0). Race was coded as a dummy

variable with the candidates that self identify as having minority status coded as minority

(1) and those who were Caucasian coded as 0. The age of the candidate at the time of

participation in the promotional process is operationalized as age in years. While age is

not always directly linked to tenure on the police force, it has been used as a proxy for the

length of tenure at LTMPD. The argument espoused by the literature suggests that age

may be a determining factor in assessment centers and promotion. In other words, a

younger candidate who is less than thirty years old will not be expected to have as many

years of tenure as a 50 year old candidate. Gender was also tested as a demographic

predictor. For the purposes of this research, female was coded as (1) and male (0).

Commendations and sustained complaints are used as measures of promotional

candidate past behavior. Commendations are arguably the results of good behavior being

rewarded and have a mean of 55.21 with a range from 12 commendations to 151.

Conversely, complaints are the results of wrongful behavior and occur after an Internal

Affairs investigation where the officer is deemed to have violated a rule. Also, by using

sustained complaints, only the most serious violations are recorded. While the actual
discipline received from a sustained complaint can range from termination to verbal

counseling, internal affairs investigates grievous acts; often sending less serious matters

back to the officer’s chain of command for a more informal review. The data indicate

that some candidates had as many as seven sustained complaints or as few as zero with a

mean of .81.

The times tested variable (coded as the number of times tested during the last 3

testing opportunities) represents the number of times a person participated in the

promotional process. This variable is included to take into account the possibility of a

learning effect as suggested by the literature. Only eight the candidates took the test each

time it was offered (total three). Interestingly, no one who participated in all three

assessment centers has been promoted. Furthermore, only nine of the forty-nine who

took the test two times were promoted. The remaining sixty-one were promoted the first

time they participated in the promotional process.

As this research is done on the amalgamation of three testing procedures, another

variable that is slightly altered (or dependent upon time tested, etc.) is tenure. (See Figure

1 for an illustration of tenure and its relationship to promotion for each of the three

testing scenarios.
Figure 1: Tenure of Promotional Candidates
Tenure as an LTMPD Sergeant
90
80
70
60
50
Time in Months Promoted
40
30 Not Promoted
20
10
0
2001 2005 2007 All Years
Test Year

Source: Open Records request via Public information Act (Texas Open Records
Act), Chapter 552, Texas Government Code (2009).

Surprisingly, there is a dearth of literature explicating the effect of tenure alone. It is

provide here in order to show that there is very little difference in the average amount of

tenure each candidate has at the sergeant rank among the promoted and not promoted.

Within this police department, this is not an unexpected consequence. The time spent as

a sergeant (the rank just below my analysis rank) is expected to have little effect on the

actual promotion because any person holding their rank for at least 6 months can enter

the promotional process for the next rank. Figure 1 represents the average tenure in

months for each test and then all tests combined: 2001 test average tenure for promoted

was 71 months, not promoted 72 months; 2005 test average tenure for promoted was 79

months, not promoted 80 months, and the 2007 test average tenure for promoted was 70

months, not promoted 67 months. The total average tenure for all three tests combined

show that overall those who were promoted had 74 months of tenure compared to an
average 71 months for those not promoted.

LIMITATIONS

One of the main limitations of this analysis is inherent of any study revolving

around one particular group of people; limited external validity. This analysis applies to

one police department in which individuals were testing for one specific rank. Increasing

this test to analyze the police candidates testing for the rank sergeant or to data outside of

the LTMPD may provide differential findings. Furthermore, this is a self selected

population and inferences concerning subordinate officers as well as those who chose not

to participate in the process cannot accurately be made.

Another limitation is the restricted number of observations from which data were

collected. Because of the limited number of minority candidates, it was not possible to

break the racial component down any further. The merging of all non-white officers into

the minority variable, for instance, was needed in order to keep the model stable.

Analysis of a larger group such as the lieutenants and sergeants may result in the ability

to tease out single race effects. It should be noted here that there was one observation

dropped from the 202 officers represented in this analysis. One person from the first

testing group retired during the process and only incomplete data were available. The

impact from dropping this person is minimal because it was a white male and, therefore a

member of two of the largest groups represented in this study; white and male.

Lastly, past behavior was represented by sustained complaints and

commendations. Proxies for past behavior are problematic because individuals could

have received a generous portion of commendations of sustained complaints AFTER

being promoted. The temporal aspect of the past behavior was not entirely gleaned from
the data. Performance reviews are available for consideration however there are several

notable problems with doing so. First, supervisors must do little documentation or a

“Meets Standards” or “Exceeds Standards”. However, for a rating of “Does Not Meet

Standards” a considerable amount of documentation must be done. Secondly, this rating

can be appealed should the person being rated feel the need to do so. Therefore, it is

profoundly unlikely that supervisors created more work for themselves when a passing

performance score could be more easily given. Secondly, a considerable amount of time

would have to be dedicated to reading each three or four page review for all 202

candidates and this is an overwhelming time constraint for this study. What is more,

when coupled with the likely invariability of the data for reasons stated above, a lengthy

review of performance plans would have likely been unproductive.

 CHAPTER FOUR

ANALYSES AND RESULTS

Stata (Statacorp, 2007) was used to perform two sets of analyses in this research

project. First, using Ordinary Least Squares (OLS) regression, this research paper

examined the effects of several important variables on High Assessment Center (AC)

Score. These variables included the following: written test results, work history

indicators, and demographics. The OLS analyses were used to test hypotheses regarding

which attributes were significantly related to candidates receiving higher overall AC

scores.

In Table 4, high AC score was the dependant variable. In this model, written high

score (β = .02) was significant at P < 0 .013. Age had a larger coefficient than written

high score (β= -.05), and was significant at P > 0.00. The commendations variable (β = .

01) was also significant, P < 0.04. The OLS regression model was able to predict 17% of

the variance found in the dependant variable which is very modest.

Table 4: Regression Model of Predictors for a High Assessment Center Score (N=202)

Coefficient Std.
High Assessment Center Score s Err. t P>[t]
Written High Score 0.020 0.010 2.490 0.013**
Education 0.070 0.140 0.500 0.620
Female 0.120 316.000 0.730 0.470
Minority -0.210 0.130 -1.590 0.110
Age in Years -0.050 0.010 -4.270 0.00***
Commendations 0.010 0.002 2.060 0.04**
Sustained Complaints -0.040 0.040 -0.870 0.390
Times Tested -0.110 0.120 -0.910 0.370
Constant 7.810 0.870 9.000 0.00***
Prob > F = 0.00, R2 = .17. Significance P < .05**, P < .01***
Source: Open Records request via Public information Act (Texas Open Records Act),
Chapter 552, Texas Government Code (2009).
Despite the literature, education, gender, and minority status failed to rise to a level of

significance and did not appear to be related to a high AC score.

Diagnostics of the OLS model run for this analysis showed the data to be

normally distributed. A variance inflation factor analysis assessed multicollinearity. The

mean VIF was 1.09 and this did not present a problem (Acock, 2008). Stepwise

regressions were performed and the model was robust.

Table 5 reports the logistic regression model utilized the same variables as in

Table 4. Promoted was the dependent variable. The AC coefficient is both positive (β =

1.04) and highly significant (p < 0.00). According to Acock (2008) this can be

interpreted as a 4 percent increase for every one-hundredth of a point, or a 48 percent

increase in odds of promotion with every tenth of a point improvement in score. The

odds ratio for written high score was 1.31 and it was significant indicating that for every

point higher in the written score, a candidate’s odds of being promoted increased 31

percent. This is not surprising and is to be expected given the equation used for the final

score: Written Score (40%) + Assessment Score (60%) = Final Score.

Table 5: Logistic Regression model of Predictors of Promotion (N = 202)

PROMOTED Odds Std. Z P>[Z] [95% Interval]

Ratio Err. Conf.
High Assessment Center 1.04 0.01 5.51 0.00** 1.03 1.06
Score *
Written High Score 1.31 0.08 4.42 0.00** 1.16 1.48
*
Education 0.47 0.34 -1.05 0.29 0.12 1.91
Female 0.78 0.69 -0.28 0.79 0.14 4.45
Minority 0.78 0.52 -0.38 0.71 0.21 2.92
Age in Years 1.06 0.07 0.82 0.41 0.93 1.21
Commendations 0.99 ..01 -0.91 0.36 0.96 1.01
Sustained Complaints 0.3 0.13 -2.74 0.01** 0.12 0.71
*
 Times Tested 0.03 0.03 -4.25 0.00** 0.01 0.16
*
Significance P < .05**, P < .01***.
Source: Open Records request via Public information Act (Texas Open Records
Act), Chapter 552, Texas Government Code (2009).
A diagnostic test was done on the logistic model by testing the area under the receiver

operating characteristic (LROC). For this model, LROC = .98 which suggests a stable

model.

The number of commendations that the promotional participant receives is

significant (p < 0.04) and positive in the AC model. These findings do not represent any

previous research findings on past behavior effects on AC’s and may warrant further

research. Additionally, the results on age provide proof of its effect on the AC rating.

Age was negative and significant (-0.05; p < 0.000) and demonstrates a candidate’s age is

inversely related to obtaining a high AC score.

Concerning past behavior, the odds ratio for sustained complaints is 0.30 and it is

significant. This result suggests that for every sustained complaint a candidate’s odds of

being promoted decrease by 70 percent. The number of times tested had a similar inverse

effect on the odds of promotion. Evidence provided by the promotion mode

demonstrated the odds of being promoted decrease by 97 percent with each additional

test taken.

For the purpose of testing both research questions (AC Score and Promotability),

the variable for candidate’s status as a minority was a dichotomized. Minority (non-

white) was coded as 1 and non-minority equaled 0. This was done to ensure a proper

number of observations in order to keep the models stable. However, Pearson chi-

squared tests were performed to test the relationship between and individual race
characteristics and promotion. In these tests, being Black was the only significant race

relationship, P < 0.02. All other races proved to have no statistically significant

relationship to promotion. Chi-squared tests showed that minority status and promotion

were significantly related at the .05 level (see Table 6).

Table 6: Chi-Squared Tests of Individual Race Characteristics and Minority Status

Chi- d.f. Prob.

Square
Black 5.285 1 0.022
Hispanic 0.337 1 0.562
Asian 0.424 1 0.052
Native American 0.210 1 0.647
Minority 4.203 1 .040
 CHAPTER FIVE

DISCUSSION AND CONCLUSION

Discussion

There are several key findings which can be gleaned from the data analysis. It

was expected that AC score would be a strong predictor of promotion; there is a 4 percent

increase in odds of being promoted with every one-hundredth of a point increase in score.

Likewise, it is not surprising that the written high score variable in the promotion model

reported a 31 percent increase in the odds of promotion. However, a 31 percent increase

in odds accounts for a whole point difference; a mere tenth of a point increase in AC

score raises the odds of promotion by 48 percent. Thus, AC score increases the odds of

promotion to a much greater extent than does the written score.

Written high score is statistically significant in predicting promotional outcome;

however, the coefficient was rather small. This suggests that even though written test

scores account for 40 percent of the overall grade given at the end of the promotional

testing process, it is the assessment process that truly influences how promotional

decisions are made. In short, doing very well on the written portion of the test makes no

“real” impact on the overall final score. Written score is merely a qualifier to move onto

the AC portion of the testing process.

The independent variables of minority status and being female failed to achieve

significance in either model. This actually indicates that the department as a whole and

possibly the AC specifically uses determinants for promotion other than race and gender.

This is often a concern with any organization; a person may be promoted because of the

“good ole boy system” or because the person was a minority or female (i.e. through
affirmative action initiatives). By controlling for these independent variables, it was

shown that race and gender have no significant influence on promotability. This suggests

to some extent that the LTMPD promotions are blind to one’s physical characteristics as

organizational theory suggests it should be.

The age of the individual is negative and significant in the OLS model but not in

the model testing its relationship to being promoted. This provides evidence of the

argument that younger promotional candidates are being sought (Adams, 2002), and

lends proof to the argument that the AC scores younger candidates higher than they do

the more seasoned veterans. Younger candidates are being scored higher than older

candidates and tenure appears show almost no differentiation between promoted and non-

promoted. This suggests there are other reasons for younger officers being promoted

other than the possibility of having more experience in the rank. This is an important

finding and should be investigated further.

Past behavior variables which were operationalized as the number of

commendations and the number of sustained complaints offered some insight.

Commendations were significantly related to high AC score but not to being promoted.

In contrast, sustained complaints did not prove significant in the OLS model but it was

significant and inversely related to being promoted. This insight is somewhat constrained

because both are subjectively applied to the officer by their chain of command. How to

interpret this remains unclear but provides some groundwork for future research on the

effects of past behavior on AC’s and promotion.

Times tested is not significant in the OLS model, however the negative coefficient

would suggest that the outcomes leans towards the notion that with every test taken the
odds of being promoted are significantly lowered. The number of times tested becomes

significant in model testing the dependant variable promoted, and indicates that repeated

participation in the entire promotional process diminishes the candidate’s odds of

promotion vastly. Additionally, the logistic regression model indicates that education

still has no significance which provides robust proof that one’s educational level plays no

role in promotability. In other words, the non-significance of the coefficient for

education in both models indicates that for the LTMPD, education is not a valued

characteristic and has no predictive effect on one’s promotion.

There is a caveat to the gender questions which is implicit in the findings.

Females comprise 18.57% of the 70 lieutenants that have been promoted over the

course three promotional processes analyzed in thesis. They make up 18.81% of

the 101 lieutenants on the LTMPD. The data suggest females are evenly

represented in promotions and are not advanced in rank at a rate greater than their

male counterparts. Again, this may suggest that the LTMPD is gender neutral

with respect to promotions. However, the data can also be interpreted as the

female promotion rates being purposefully kept within a specific percentage.

Having some college education has no predictive effect on one’s AC score

or promotion. Furthermore, the level of advanced education either does not help

one to obtain a high AC score. Additionally, the number of times tested appears

to hurt one’s chances of promotion in the logistic regression model but has no

effect on a high AC score in the OLS model. This would suggest that feedback

from prior tests, contrary to the literature (Kroecker, 2000), has little or no impact

on subsequent AC outcomes and numerous attempts at getting promoted

significantly decrease the odds of it occurring. Furthermore, there seems to be no

difference in the amount of tenure among those promoted and those not promoted.

Conclusion

The results from this study offer other intriguing insights for answers to the

question of what it takes to be promoted. Currently, there are just over 100 officers at the

rank of lieutenant within this police department, including the 70 that have been

promoted since November 2001. It is one of the top ten largest metropolitan police

departments in the United States (Hickman and Reaves, 2006); serving more than

1,000,000 people and seemingly has a large turnover of its management level officers. In

a metropolitan police department of this size and given the importance of this upper-

management position, there can be little argument against efficient, thorough and,

repeated testing of the effectiveness and efficiency of the promotional processes.

This current analysis of the testing procedures in the LTMPD answers the basic

question; what does it take to get promoted? It has been empirically shown that doing

well in the AC strongly affects final scores. Because of the strength of the AC score on

final score, AC high score predictors become highly important.

The significance in the findings provided by the AC model are vividly apparent

when one recognizes there are no known validity tests of the AC process done by the

LTMPD. Experientially known, the AC proctors go to great lengths to ensure the

candidates “stay in character” when addressing the assessors suggests that role playing is

a significant part of the process. It would appear that the ability to play the role is the

most important aspect of the AC, a point well made by Lance (2008).

An aggregate view of the data strongly suggests this department lends little or no
value to maturity (age), experience (tenure), education (college hours), or perseverance

(times tested). This is astounding considering that the rank of lieutenant is crucial to the

successful relationship between the organization and its personnel as well as the

organization and the society it serves. However, this research paper does not argue for

the abandonment of the AC process. Indeed, continuing to utilize the AC would be

advantageous. Using the AC as one of many tools in the promotional toolkit and would

certainly be useful for determining performance and capabilities of the promotional

candidate. Nevertheless, holding out AC rating as the best, stand alone indicator of

future performance is highly unrealistic.

Considering the importance of organizational theory of promoting the best to

guarantee organizational success and the theory of police behavior offered by Klinger

(1997; 2001), promotion of those who are most likely to be successful is absolutely

necessary. That is not to say those currently in leadership positions on the LTMPD are

not the best and brightest; the fact is there is no way to know at this point. However, we

now have a clearer understanding of the promotional process and what it takes to get

promoted in the LTMPD. Predictors of promotion have been empirically tested and lead

to a question ripe for further analysis. Explicitly, do those skills, behaviors, or

characteristics that it takes to get promoted on the LTMPD positively correlate with what

it takes to be a successful leader? Tacitly, this current research suggests promotions are

serendipitous and not related to true ability beyond one’s stellar performance at the AC,

as well as having the good fortune to have lenient or favorably assessors.

American police agencies are paramilitary organizations (Jermier and Berkes,

1979) in which the rank of lieutenant is a position of leadership. Several predictors of

promotion were uncovered and provide enlightenment into the behavior and structure of

this police organization. Through the data analysis, we see that the command structure is

driven almost entirely by AC score. Simply stated, a high AC score significantly raises

the odds of being promoted to lieutenant. The LTMPD’s chief level officers are chosen

from the lieutenant ranks. Because of this, the variables affecting the AC score become

the most important when discussing organizational behavior and its effect on police

personnel as well as on society.

In light of the importance of promotion to the lieutenant rank and the financial

cost of using the AC, the implications of the findings is glaring. Fervidly suggested by

the current research is that organizations utilizing and heavily relying on AC’s better be

absolutely convinced of their ability to predict future leadership behavior. It will be those

AC scores that greatly influences the list of who is worthy of promotion. From that list,

leaders will be chosen and they will contribute substantially to the societal and

organizational success (or failure) of their police department.

 ENDNOTES

1. All data presented for written and assessment center test scores, assessment center
attendance dates and overall scores as well as all candidate demographic
information were obtained by way of open records request sent in accordance to
the Public information Act (Texas Open Records Act), Chapter 552, Texas
Government Code. This law was extensively used and is imperative in order to
legally and ethically acquire the reported data.

2. The combined cost to the LTMPD for all three assessment centers was in excess
of $100,000.

3. There is an argument that Assessment Centers may indeed be able to accurately

predict successful leadership behavior, and thus are valuable tools in the
promotional procedures of police departments (Ross, 1980; Kroecker, 2000).

4. The tactical exercise is one that includes a scenario in which the candidate would
be expected to apply his/her knowledge of street tactics such as crime scene
protection, mass arrest or large accident scene. The oral presentation is one in
which the candidate’s scenario would be a situation in which the candidate would
have to act as if they are giving a speech to the city council, command staff, etc.
An in-basket exercise simulates a full in-basket in which there are multiple
scenarios ranging from personnel issues to complaints against officers or crime
reduction operations. The candidate is expected to address and prioritize each
incident.

5. Kroecker’s article appeared in The Police Chief and was written for chiefs of
police who would be assumed to have been benefactors of the AC and therefore
more inclined to promote its use.

6. The LTMPD hires people as young as 19 ½ years old with the condition that they
have at least 60 hours of credit at an accredited college. Once hired a recruit must
endure a 32-week academy training program followed by 24 weeks of field
training. After field training, the recruit is in probationary status for another six
months. Therefore, a person would be 21 before being a full, non-probationary
police officer.

7. All candidates with a written score at or above the cut off score was allowed to
attend the assessment center for that particular test:

a. Test 1cut off: score of 61.

b. Test 2 cut off: score of 57.

 c. Test 3 cut off: score of 62.

APPENDIX

The formulas used by the LTMPD to determine the candidates’ positions on the

final list are as follows:

Test 1- November 1, 2001 Statistical Formulas

Written score:
(((Raw Written Score-74.344)/7.524) x 5)+85)=STANDARIZED WRITTEN SCORE
Assessment score:
(((Raw AC Score-7.552)/.721) x 5)+85)=STANDARIZED AC SCORE
Final Score:
(Stnd. Written x .40)=( Stnd. AC x .60)=FINAL SCORE

Test 2- October 20, 2005 Statistical Formulas

Written score:
(((Raw Written Score-67.33)/7.068) x 5)+85)=STANDARIZED WRITTEN SCORE
Assessment score:
(((Raw AC Score-3.409)/.0561) x 5)+85)=STANDARIZED AC SCORE
Final Score:
(Stnd. Written x .40)=( Stnd. AC x .60)=FINAL SCORE

Test 3-October 30, 2007 Statistical Formulas

Written score:
(((Raw Written Score-73.705)/7.826) x 5)+85)=STANDARIZED WRITTEN SCORE
Assessment score:
(((Raw AC Score-3.445)/.447) x 5)+85)=STANDARIZED AC SCORE
Final Score:
(Stnd. Written x .40)=(Stnd. AC x .60)=FINAL SCORE

As can be seen here, the denominator used in the written score and assessment

center score calculations vary greatly. The basic sequence is the raw score-average and
the result is divided by an unknown number. The product is multiplied by 5 and added to

85. This researcher was unable to find the reason for the disparity however one scoring

model remains consistent. The final score calculations are 40% written plus 60%

assessment center. It is easy to see here why the assessment center has such a large effect

on the final outcome. Not only is it afforded a larger percentage of the overall score, it is

a much smaller number in the standardized formula compared to the written score. Any

variation would have a much greater influence in the final outcome. As a result of those

final outcomes, a hierarchical list is produced with the top score as number one, the next

score as number two and so forth.