Examining Applicant Reactions to Different

Media Types in Character-based Simulations

for Employee Selection

Valentina Bruk-Lee*, Julie Lanz*, Erica N. Drew*,Chris Coughlin**,†, Pamela Levine**,†

, Kathy Tuzinski**,† andKimberly Wrenn**,†

*Florida International University

International Journal of Selection and Assessment ，

2016
YING LIU

2023.06.18
 6.4.2 情境判断测试
1. 告诉这位顾客直接去亚马逊网站购买这
情境判断测试是一种“用来评 款手机。
价一位求职者对工作场所中可 2. 告诉这位顾客稍等 20 分钟，等你处理

能会碰到的某种情境作为的判 好其他顾客的问题再回来接待他。
3. 告诉这位顾客在距这里大约 30 分钟车
断”的人事判断。
程的地方还有一家百思买的商店，那里有

情境案例： 价格较低的类似手机。
假如你要应聘百思买公司销售员 4. 告诉这位顾客在本地的 Sprint 手机经销
一位顾客带着一张从亚马逊网站下载打 商那里还能买到甚至比亚马逊价格更低的
印的三星 Galaxy 手机资料向你走过来。
手机。
然后他向你详细地询问有关价格、电池
寿命以及如何使用这款手机等方面的信 5. 向这位顾客解释说本店还有一些能够更
息，同时还向你提到“这款手机在亚马 好地满足客户需求的类似手机，并且说明
逊上的销售价格比你们这儿要便宜 50 这种手机的优点。
美元呢”。现在，你已经在这位顾客身
6. 请你的主管过来，努力劝说这位顾客从
上耗了将近一小时的时间，而此时还有
其他顾客在等待你提供服务。 你这里购买这部 Galaxy 手机。
这时你将会做的事情是：
6.4.4 情境测试和视频情境测试 6.4.5 小型工作培训和评价法
情境测试（ situational test ）： 小型工作培训和评价法（ miniature
要求被试对能够代表工作场景的典
job training and evaluation ）：
型情境作出反应。工作样本法、评
指在雇用求职者之前，首先培训他
价中心中需要完成的任务（比如公
们去完成几项工作任务，然后对他
文筐练习）、视频测试、小型工作
们的绩效进行评价。该方法与工作
培训都属于这种类型的测试。
视频模拟测试 样本法类似，使用实际的工作样本
（ video-based simulation ）： 对求职者进行测试，测试的内容本
通过在线的方式或电脑播放的方式 质与职位相关，效度较高。
向被试展现几个视频情境，在每个 本田汽车公司案例：
情境后面都有一道或多道选择题。 招募求职者→筛选求职者（住址、
教育背景、经验等）→每周培训→
6.4.6 实际工作预览 专家打分→公司人力资源部门面试
现实中的工作是最好的筛选工具。 + 审查培训记录→确定录用人员→
比如向求职者直接说明工作时间 新员工接受一项一次性药物测试→
表，询问他们的工作偏好，降低 入职培训→日常培训
离职率。
Examining Applicant Reactions to Different
Media Types in Character-based Simulations
for Employee Selection

Valentina Bruk-Lee*, Julie Lanz*, Erica N. Drew*,Chris Coughlin**,†, Pamela Levine**,†

, Kathy Tuzinski**,† andKimberly Wrenn**,†

*Florida International University

International Journal of Selection and Assessment ，

2016
YING LIU

2023.06.18
 Motivation& Research Question

Assessment ： Static Text Based Paper-and-Pencil

Interactive and Media-Rich, Making Computer-Based Testing (CBT)

Enhanced Realism, Increased Complexity, and Higher Engagement

Research ： Influence of technology and medium of

assessment administration on applicant reactions

media types in employee character-based simulations ？

Research Question
A Series of Two Studies
Hypothesis STUDY1

Hypothesis 1 (H1): Perceived job relatedness will be significantly higher for a 3D animated computer-
based SJT than for a text computer-based SJT of the same content.

Hypothesis 2 (H2): Perceptions of opportunity to perform will be significantly higher for a 3D

animated computer-based SJT than for a text computer-based SJT of the same content.

Hypothesis 3 (H3): Applicant perceptions of a company will be significantly more favorable for a 3D
animated computer-based SJT than for a text computer-based SJT of the same content.

Hypothesis 4 (H4): Procedural justice perceptions of (a)job relatedness and (b) opportunity to perform
of the3D animated computer-based SJT will positively predict unique variance in company perceptions.
Hypothesis STUDY1

Hypothesis 5 (H5): Overall procedural justice perceptions will mediate the relationship between (a)
job relatedness and company perceptions, as well as between (b) opportunity to perform and company
perceptions for a 3D animated computer-based SJT.

Hypothesis 6 (H6): Perceptions of (a) job relatedness and (b) opportunity to perform for a personality
assessment will be significantly higher when included in a test battery containing a 3D animated
computerbased SJT than a text computer-based SJT of the same content.

Hypothesis 7 (H7): Perceptions of (a) job relatedness and(b) opportunity to perform for a cognitive
ability assessment will be significantly higher when included in a test battery containing a 3D animated
computer-based SJT than a text computer-based SJTof the same content.
Hypotheses STUDY1

Procedural justice
perceptions

H4

Figure 1. Hypotheses Framework

Method STUDY1
Participants

-Four hundred and forty undergraduate students from a large research university in
the southeast participated in a simulated hiring process for a retail position.

Approximately half of the participants reported being currently employed (46%).

Participants who reported having experience in retail (48%) had an average tenure of
2.42 years. $50 gift card
Method STUDY1
Procedure
-Participants were given an assessment battery including a set of proprietary
measures that are commonly used to make hiring decisions (a SJT, personality
measure, and cognitive ability test).

-Using a within-subjects design, the assessment battery was comprised of two

conditions: (A) a media-rich (3D) computer-based SJT, a personality test,and a
quantitative cognitive ability test; and (B) a text computer-based SJT, a personality
test, and a quantitative cognitive ability test.

-Participants completed measures of job relatedness and opportunity to perform

immediately following each of the three assessments.

-Participants completed either A or B at Time 1, and 7–10 days later were contacted
to complete Time 2 (participant response rate at Time 2 was 58%).
(AB=225, BA=215)
Method STUDY1
Measures

Situational judgment tests

Both Versions
Eight Scenarios

For example, in one scenario, an employee is on the phone

helping a female customer locate a pair of glasses at another store,
and a male customer asks to be helped. The female customer
rudely tells the second to wait his turn. In response to the
scenario, participants were asked to indicate which of four
behavioral options would be most and least effective to employ.
Method STUDY1
Measures

Situational judgment tests

Personality

Cognitive ability test

Job relatedness
5-point Likert scale
Opportunity to perform 1= strongly disagree
5=strongly agree.

Overall procedural justice

Company perceptions
Results STUDY1

H1 √
H2 X
H3 √
H4 √
Results STUDY1

H5a √

H5b √
 STUDY1
Results
Perceived Job Relatedness

3D SJT > TEXT SJT H1 Perceptions of Opportunity

H2 to Perform

H3 Applicant Perceptions
Media Types of a Company
H6
Perceptions of Personality
H7
Assessment

Perceptions of Cognitive
H5a
ability assessment
Job Relatedness
P5(+)
Procedural justice H4
Company Perceptions
)
P6(+
perceptions
Opportunity to Perform
H5b
Research Question
A Series of Two Studies
Hypothesis STUDY2

Hypothesis 1 (H1): There will be significant differences in ratings of job relatedness across media
types (2D animation, 3D animation, or video) using a computerbased SJT of the same content.

Hypothesis 2 (H2): There will be significant differences in levels of engagement in the application
process across media types (2D animation, 3D animation, or video) using a computer-based SJT of the
same content.

Hypothesis 3 (H3): There will be significant differences in rankings of (a) preference, (b) realism, and
(c) overall company impressions across media types (2D animation, 3D animation, or video) using a
computer-based SJT of the same content.
 STUDY2
Hypotheses

2D animation Ratings of Job Relatedness

3D animation H1
video Engagement in the
H2
Application Process

H3a
Rankings of Preference
Media Types H3b

H3c Rankings of Realism

Rankings of Overall
Company Impressions

Figure 2. Hypotheses Framework

Method STUDY2
Participants

-Participants were recruited through an open access convenience sampling website

-Of the 434 individuals that read the informed consent and started the survey, 209
participants completed the study.

-Of these, three were removed for spending less than 5min on the survey and four
participants were removed for spending 15 or more hours.
Method STUDY2
Participants

-
Method STUDY2
Materials

-Computer-based scenario

Three types of the same computer-based scenario were used in this study:

(a) 2D animation;
(b) 3D animation;
(c) live-action video.

In response to the scenario, participants indicated which behavioral options would

be:

(1) most
(2) least effective for an individual to employ.
Method STUDY2
Procedure

-Participants were provided with a description of the scenario to provide them with
situational context.
-Participants were asked to imagine they were applying for a job as a manager of a team
in a call center and told that the role would involve some degree of coaching.

-In the scenario, the manager had to inform his or her teammates that their customer
service goal had increased from 89% to 92%.
-Participants were asked to imagine how they would respond if they had to coach the
team.
Method STUDY2
Measures

5-point Likert scale

Job relatedness
1= strongly disagree
5=strongly agree.
Engagement

Applicant reaction rankings ：

-Preference was measured with the following item: ‘Which test would you most
prefer to take?’
-Realism was assessed by asking participants ‘Which test seems the most
realistic?’
-The overall impression of the organization was measured by asking ‘Which
test conveys the best impression to job candidates about the hiring organization?’
Results STUDY2

H1 √
H2 √
Results STUDY2

H3a√
H3b√
H3c√
 STUDY2
Results

2D animation Ratings of Job Relatedness

3D animation H1
video Engagement in the
H2
Application Process

H3a
Rankings of Preference
Media Types H3b

H3c Rankings of Realism

Rankings of Overall
Company Impressions
Discussion on the results STUDY2

In a series of two studies, the authors first investigated the benefits of using 3D animation
over text, and later explored differences in reactions across multiple forms of widely used
media.

The findings generally indicate that companies using 3D animation and live-action video
media content in their character-based simulations may reap the additional benefits of
engendering favorable reactions to the selection process.

However, the choice of media use and type is clearly one that must be weighed in relation
to practical considerations, such as cost, time, development resources, and customer
preferences.
Limitations of the study

Our results in Study 1 were limited to differences in post-test reactions only when testing
for support of a spillover effect of favorable reactions generated by the use of 3D animation
on the other assessments.

Future research should consider the inclusion of pretest reactions as well, which could be
used for comparison within each media condition.

Study 2 was limited to exploring differences across media types in reaction ratings and
rankings. While this fulfilled the study’s purpose of identifying preferences for specific
media types, they did not evaluate mechanisms by which these reactions could impact
organizational outcomes as was done in Study 1.
Directions for future
research
Generational differences in preferences for the use of technology have been widely cited.

What influence do these preferences and usage habits have in forming attitudes and reactions
about the use of media-rich assessments?

Can the attributes and use of animation present a confound in applicant responses to various
test types and item formats?

Do applicant preferences for media richness vary by the type of construct measured?

Furthermore, the authors encourage researchers and practitioners to consider the literature in
areas such as informatics and computing when studying the influence of media on employee
assessments.
STUDY1

THE

END

STUDY2
Examining Applicant Reactions to Different
Media Types in Character-based Simulations
for Employee Selection

THANK YOU

YING LIU
2023.06.19