Assessment in Industrial

Setting
Current Trends and Issues

Darwin C. Macalanda, PhD. RPm, Rpsy, CSAP

Psych Systems and Innovation
Understanding Behavior in Organizations
• The field of industrial and organizational psychology
is generally concerned with understanding and
predicting human behavior in work organizations.
• As such, its practitioners are involved in the use of
assessments aimed at characterizing or describing
the status of individuals (e.g., workers or managers),
collections of individuals (work groups or teams),
and/or organizations.
• Moreover, these assessments can be with regard to
attributes, processes, dynamics (changes), or level of
effectiveness.
Use of Assessment in Industrial Setting
• Assessments can be conducted for purposes of
categorization or description, as in the case of
obtaining insights of a human resource management
policy or practice. Thus, we might assess the
proportion of workers who are dissatisfied at work as
input to a management information system.
• Assessments also might be used for making or testing
predictions, as when a consultant is interested in
testing a theory of the antecedents or consequences
of job satisfaction.
• Assessments are used as a diagnostic tool to determine
the need for or the nature of the most appropriate
intervention.
• Assessments are typically made with regard to
interventions.
• An assessment might be carried out to monitor the
progress or the final impact of an intervention once
initiated.
• As noted, assessments in industrial and organizational
psychology can focus on the individual, the work group
or even the organization.
• Furthermore, the nature (content) of an assessment can
be with regard to attributes (of individuals or teams or
organizations), to processes (their behavior) or to
outcomes (the result of behavior).
• Assessments will usually be selected with reference to
some purpose. Some assessments will be aimed at the
measurement of a construct or variable (for research),
others at diagnosis.
• But frequently measures (especially at the level of the
individual) are gathered for purposes of prediction (e.g.,
of future job performance).
Major Forms of Assessments
Found in the Field.
 Assessments of the Individual
• Most assessments of individuals are obtained from
observations and judgments made by trained staff in
standardized settings.
• Assessments here are usually made with the goal of
establishing the person’s status relative to a
construct of interest (Murphy, 1996) or for the
purpose of taking administrative (personnel
selection) action (Schmitt & Borman, 1993) or
providing a developmental intervention.
• Traditional examples include assessment centers, job
sample (performance) tests or work simulations.
• The structured interview and the psychological
assessment interview should also be highlighted.
Furthermore, the latter can involve the use of more than
one participant-observer as in the form of a panel
interview.
• Observations of individuals are often obtained in
naturalistic contexts as well. In this regard, assessments
can be based on ratings from supervisors, peers, or
subordinates, and even customers.
• When these are obtained from several points of view on
the same or similar instruments and at the same time, the
assessments are commonly referred to as multisource or
3600 ratings. These too can be with regard to such things
as a person’s attributes (traits), typical behaviors, levels of
performance, or potential.
• A second prevalent approach to assessments at the
individual level involves standardized tests.
• These usually are of two general types. Cognitive
ability tests usually are aimed at estimating the
achievement or capacity for cognitive functioning in
one or more domains (e.g., the amount or use of
knowledge, verbal or math reasoning. or judgment).
• On the other hand, tests may be used to assess
aspects of personality, that is, job-related traits or
qualities, expected behaviors, mood disposition,
needs, or values.
• Most tests require the individual to respond to
written material presented in a paper format. But
increasingly, tests are computer administered and
scored.
• Individuals may be assessed by way of other types of
self-reports, such as a biodata inventory.
• This involves responding to a series of questions relative
to one’s developmental experiences as these would
produce or would reflect certain qualities needed for job
success.
• A special version of this is what has become referred to
as an honesty test or integrity test (Ones, Viswesvaran, &
Schmidt, 1993).
• This type of inventory contains a mixture of background
and values items.
• Finally, the individual may be assessed with regard to
products or artifacts of behavior.
• Approaches here can range from the use of records
such as employee attendance data, the handwriting
analysis of scripts (also known as graphology), the
tracings of a polygraph machine in response to a set
of “guilty knowledge” questions, and drug testing
based on blood or urine samples.
 Assessments of
Work Teams and Groups
• There are three principle approaches to assessment in this
area and parallel those aimed at collecting information on
individuals: observations, ratings, and an analysis of
artifacts.
• Insights about teams are usually based on observations in
standardized settings, often in training simulations or low-
fidelity simulations, such as computer-mediated business
games or, as in the military, war games.
• In actual work settings, supervisor or customer ratings
might be used. In both contexts team member ratings
might also be obtained to get at those aspects of team
functioning of interest.
• In actual work settings, supervisor or customer ratings
might be used. In both contexts team member ratings
might also be obtained to get at those aspects of team
functioning of interest.
• Finally, the artifacts of team and team member behavior
might be assessed.
• Included here is the content analysis of team processes
based on recorded verbal or written communications
patterns and team activities and the use of team
performance measures.
• In general, the valid assessment of teams requires that it
be done over some extended period of time or across
assignments or tasks (O’Neil, Baker, & Kazlauskas, 1992).
 Assessments of Organizations
• The assessment of organizational level phenomena will
also depend on the purpose or the goal of the
investigator.
• However, in all likelihood, it will involve the systematic
gathering of information from current employees. This
can take the form of key informant interviews, including
contacts with the chief executive officer (CEO).
• Focus groups made up of six to ten employees might be
convened.
• Far more frequent, however, is the use of organizational
climate surveys to gain insight as to aspects of company
climate or culture.
• For some applications, surveys may be aimed at obtaining
the views of customers, suppliers, or shareholders as well.
• As in the assessment of teams, the assessment of
organizations may also involve the analysis of artifacts. This
includes the content analysis of written or electronic
communications ( policy manuals), or an analysis of
practices with regard to the built environment (the size
and location of office space given to employees)
• Finally, an organizational assessment could rely heavily on
an analysis of company performance data (growth in
sales), including indices of financial viability (ROI).
 Some Cautions
• At this point in time there is no single or standardized
approach to assessments for purposes of organizational
research or practice.
• While there is an evolving consensus regarding the better
psychological tests available for a given application,
operationally most of the approaches to assessment noted
above should be viewed as labels representing a class of
specific forms.
• They also imply existing protocols or prototypes. Thus,
for example, with regard to a structured interview, the
field offers a great deal of guidance about appropriate
features, but professional judgment and local empirical
work would still be required to implement an interview
as an assessment device for a given purpose (for
personnel selection) in a particular organization
(Campion, Palmer, & Campion, 1997).
• Another aspect of assessments in industrial and
organizational psychology worth stressing is that even
well-designed tools for assessment typically require skill
and experience in order to obtain valid data or insights.
• This implies that administrators, consultants,
practitioners, or researchers would have to demonstrate
their capacities in the correct use of a tool. Moreover,
this should be done relative to a particular setting, and
for a particular application, before the information
gathered can be treated as credible.
Trends in Industrial Assessment
• Psychometric testing has grown tremendously, and is now
entrenched in employee selection and development.
• The widespread use of psychological testing is now being
followed by more sophisticated and in-depth hiring
proccess.
• Research on the uses for and outcomes of workplace
testing continues.
• Tests themselves are becoming more accessible, and there
is a new focus on the individual’s reaction to assessment.
• New technology may well change the way workplace tests
are administered.
An extensive and widely varied
range of assessments is now
available on the market.

These may be useful in identifying

work roles and environments that
optimally suit an individual, or
provide insight for personal growth,
well-being and performance.
 Tests of abilities
• Tests of abilities are now a mainstay in the realm of
psychometric testing, covering the traditionally
measured verbal, numeric and logical areas, as well
as specialized skills.
• There are many different formats for reports being
offered by publishers today. Depending on the test, it
may be possible to obtain combined results for work
team members, or multiple ratings of an individual,
as with 360 degree feedback.
 Test of States and Traits
• There is a growing emphasis on more in-depth
assessment including tests of personality, strengths,
and emotional intelligence.
• These may be useful in identifying work roles and
environments that optimally suit an individual, or
provide insight for personal growth, well-being and
performance.
• Depending on the test, it may be possible to obtain
combined results for work team members, or multiple
ratings of an individual, as with 360 degree feedback.
Choice of which instrument to use may be influenced
by the report options available.
 Test Report Format

• There are many different formats for reports being

offered by test publishers today.
• Choice of which instrument to use may be influenced
by the report options available.
Validity and Reliability of Tests

Knowing how well tests predict work

behaviours is at the heart of
justifying psychometric assessment
for employment decisions.
• The kinds of questions being examined include how
well ability and personality tests related to
competencies like problem solving, risk taking,
creativity or motivation.
 Computer-Assisted Testing
• Although 20 years of research demonstrated that
rigorously designed computer-administered tests were
equivalent to or superior to paper-and-pencil tests, the
developers of most Internet or Web-based tests have
given little thought to equivalence (i.e., Internet or Web-
based tests have not been rigorously designed).
• Consequently, substantial differences might exist
between tests delivered on a PC and those delivered
through the Web. These differences can affect the
standardization and validity of some tests.
 Some of these factors include:
• Different browsers use different settings for fonts, colors,
and other display characteristics to deliver Web-based
tests. These potentially render a given question
differently to different examinees.
• In addition, differences in screen size and resolution
reduce the equivalence of Web-delivered tests to PC-
delivered tests.
• On a PC, the test administration software standardizes the
display for all examinees, and a standard monitor can be
used throughout a testing room.
 Some of these factors include:
• Web access and response time vary greatly from question
to question. Some of the factors that affect response time
include the speed of the examinee’s connection and the
amount of traffic on the Web at the instant the examinee
responds and receives a new question.
• Response time is further affected by the speed of the Web
server and the other demands on the Web server. For CATs,
the computational server time necessary to estimate trait
level and select the next question is yet another factor that
affects response time.
• By contrast, on a PC, only a single person is being tested at
a time and between-question response time is virtually
instantaneous, thus better standardizing test delivery.
• When tests are administered in an uncontrolled
environment, such as might occur with Web delivery,
environmental variables present during test delivery can
affect the test performance of individuals.
• A basic principle of well-standardized testing is that paper-
and-pencil tests are to be administered in a quiet and
comfortable environment.
• For the most part PC-based tests also have been
administered in testing rooms with a carefully controlled
environment.
• When tests are delivered through the Web, however, a wide
variety of extraneous factors might be present that interfere
with—and potentially invalidate—the resulting scores.
• In addition, when individuals take tests without supervision,
it is impossible to know who is actually taking the test, what
materials they were accessing during test administration,
and who was assisting them during testing.
 The potential of Gamification

•The gamification approach can provide a

thought-provoking journey through a candidate’s
aspirations and interests around job roles and
career paths. The potential for measuring areas
such as risk taking, decision making and
cognitive abilities in an engaging and interactive
way holds great potential.
What is gamification in recruitment?
• Gamification in recruitment is all about gamifying the
components of the hiring process – most notably the
assessment stages. But let’s be clear.
• This is not about introducing frivolous games to the
hiring manager’s decision-making process. Nor is it
about making tests or assessments out of games
(otherwise known as game-based assessment).
• Gamified assessments take robust, scientific
psychometric tests and introduce some elements of
gaming (such as progressing through levels, or earning
points, or getting badges to create a more engaging
and contemporary looking online test.
 Why is there so much talk about
gamification in recruitment?
• Quickly spotting the best in your applicant pool,
engaging with your candidates and making sure you
stand out from your competition are becoming
increasingly essential for all recruiters.
• Graduate-age candidates tend to expect stronger
engagement with them as individuals, immediate
feedback and a selection process that is reflective of
the mobile world in which they operate.
• Taking all this into consideration, hiring companies
want an update type of psychometric tests that
gives them the robust information they need about
the capabilities of the applicants but a fantastic user
experience for the candidates.
 What are the upsides to
gamification in recruitment?
• Stronger engagement with applicants and a reported better user
experience.
• A differentiated selection process. Candidates remember the
company as having an innovative approach to assessment.
• A hiring tool that supports any corporate, forward-thinking,
market-leading brand value.
• Supports the desire to create a more diverse and inclusive
applicant pool.
• Gamified assessments are designed for, and delivered to,
mobiles and integrate seamlessly with their application systems
• Generates more applicants – and higher quality applicants.
 And what are the downsides of
gamification in recruitment?
• Too much ‘gamification’ might give the
impression to candidates that the application
process isn’t a serious business.
• Focusing on the user experience may mean
that you lose site of the real value of the tool
– the assessment and the information this
gives you to make a decision.
Gamified assessments compare to
those from traditional online
psychometric tests?
• Research* shows that there is no significant difference
between the levels of ability measured by more
traditional online tests, and those that have included
gamified tests.
• Test takers that enjoy ‘games’, rate the hiring
organization more positively as a prospective employer
than those completing traditional online tests.
• Using gamified assessment in hiring, offers an
opportunity to only go up in the perceptions of the
applicants.
 Where does gamified assessment
fit in the hiring process?

• Attraction – pre-application, realistic job previews

have embraced gaming elements for a number of
years. They help promote the hiring company’s brand
and give an insight to the job itself for the potential
candidate. Typically, they include elements of a game
such as instant feedback and scores, but for the
benefit of the potential applicant and not the hiring
company.
• Early in the process – sometimes known as screening
applicants. Here the online gamified assessment is
presented to all applicants early on. The
assessments’ ease of use and completion, speed and
powerful information-giving qualities make it perfect
for offering to many to get a solid initial view of their
likelihood at success in your business.
• Rolls-Royce deployed a gamified assessment based
around typical situations or scenarios in its business.
Candidates were asked to make judgments about
what to do via a messaging app style of
communication via their mobile. It gave Rolls-Royce a
robust situational judgment score, and the
candidates a strong insight into the role.
• Assessment Centre stage – Assessment centers have
been paper-heavy for decades. Gamified assessment
now enables these selection events to become
paperless, more efficient and more objective.
Darwin C. Macalanda, PhD. RPm, Rpsy, CSAP
Psych Systems and Innovation
 References
• Campion, M. A., Palmer, D. K, & Campion, J. E. (1997). A review of structure in the selection interview. Personnel
Psychology, 50, 65-72.
• Coovert, M. D. (1995). Technological changes in office jobs: What do we know and what can we expect. In A. How-
ard (Ed.), The changing nature of work (pp. 175-208). San Francisco: Jossey-Bass.
• Murphy, K. R. (1996). Individual differences and behavior in organizations. San Francisco: Jossey-Bass.
• Schmitt, N., & Borman, W. (1993). Personnel selection in organizations. San Francisco: Jossey-Bass.
• Smither, R. D., Houston, J. M., & McIntyre, S. D. (1996). Organizational development. New York: HarperCollins.
• https://recruiterbox.com/blog/gamification-in-recruiting.
• https://psychology.iresearchnet.com/counseling-psychology/personality-assessment/computer-assisted-testing/
• http://talegent.com.ph/blog/the-future-of-assessment-in-the-asean-integration/
• https://insights.humancapital.aon.com/talent-assessment-whitepapers/gamification-in-assessment-upgrade-
your-talent-strategy