Selection Procedure Technical Report: December 10, 2018

Selection Procedure Technical Report
Prepared for: Shared Vision Psychological Services

Created by: Barbara Berent, Nick Jochim, and Sara Sepkowski
December 10, 2018

Selection Procedure for Shared Vision Psychological Services
Team InnovatI/On
2018
Executive Summary
OVERVIEW
Shared Vision Psychological Services (SVPS) provides quality mental health care to children,
adolescents, and adults by offering a wide range of clinical services to the community including
individual therapy, couples therapy, family therapy, group therapy, and psychological
assessments. SVPS was in need of a selection procedure to identify job candidates who possess
strong interpersonal skills. Interpersonal skills are vital to creating safe, authentic, and
communicative therapeutic relationships with clientele. Additionally, SVPS is dedicated to hiring
clinicians who have a strong understanding of the theory behind and implementation of
psychodynamic therapy, as this is the approach to care in this clinic. Due to the wide array of
therapy approaches that can be taught in psychology doctoral programs, SVPS was in need of a
selection tool to help identify qualified candidates who possess this specified knowledge and are
able to make clinical decisions and develop treatment plans from this approach. The selection
procedure developed by Team InnovatI/On meets both of these needs.
SELECTION PROCEDURE COMPONENTS
Once job candidates resumes have been screened for the minimum school requirements
including a completed clinical externship, clinical internship, and Clinical Psychology Doctoral
Degree, they will be contacted to participate in the following:
1. Structured Interview
Job candidates will partake in a 45-60 minute structured interview. This interview will be
completed by three senior staff clinicians and consists of 7 questions total. Each of these
questions are tailored to elicit responses that allow interviewees to demonstrate specific
interpersonal skills. Each question is accompanied by a score sheet for interviewers to
rate the interpersonal skills demonstrated as well as the content of their responses.There
are three additional score sheets for overall interpersonal communication skills. Each
score sheet rating is turned into a percentage that falls into one of three scores that
interviewees can receive per question/communication skill: Exceptional Performance (5),
Average Performance (3), and Poor Performance (1). These 10 scores will be summed for
a total structured interview score. The highest possible score a candidate can receive from
1
each interviewer is 50 and the lowest is 10. Scores from all three interviewers will be
averaged to determine the candidate’s final structured interview score.
2. Situational Judgement Test

The Situational Judgement Test will take place immediately following the structured
interview. Interviewers will leave the room and give the candidate 60 minutes to
complete the test. The Situational Judgement Test consists of five vignettes (50-500
words each) that demonstrate a realistic clinical case with specific clinical issues to be
addressed. Job candidates will have to create a written response that details how they
would approach the situation and determine the appropriate treatment plan. An anchored
rating scale will be used in addition to multiple raters to score candidate responses. Each
vignette response receives two anchored rating scale scores: one for appropriate clinical
analysis and one for development of a treatment plan using the Psychodynamic approach,
for a total of 10 scores. Anchored rating scale scores are: Exceptional Performance (5),
Average Performance (3), and Poor Performance (1). The highest possible score a
candidate can receive, on each test, is 50 and the lowest is 10.
SELECTION DECISION
SVPS SMEs will review resumes to determine qualified candidates to interview for the Clinical
Psychologist position. SMEs will be looking for individuals that have completed their PsyD. or
PhD. in Clinical Psychology, as well as experience/knowledge in psychodynamic theory.
Structured Interview Score + Situational Judgement Test Score = Candidate Total Score
(50 Points) (50 Points) 100 Points
The Structured Interview and Situational Judgement Test will both account for 50% of the
candidate’s total score. This was a topic of great discussion amongst Team InnovatI/On and the
SVPS SMEs.
Seventy points is the minimum recommended score to be considered for hire in this selection
process. The score of 70-79% is classified as “Average Performance”. Because of this, a score
of 70 is the lowest score a candidate can receive and still be considered qualified to fill the
position of Clinical Psychologist.
Total scores from the Structured Interview and the Situational Judgement Test will be summed to
determine the Candidate Total Score. The highest possible score is 100 and the lowest is 20.
2
Table of Contents
Component Page Number
Overview 1
Structured Interview Development and Rationale 4
Interview Format and Questions 5
Rating and Validation 6-7
Situational Judgement Test 8
Rating and Validation 9-10
Selection Decision 11
Legal Guidelines 12
Project Cost 13
Appendix 15-38
References 39-40
3
I. Structured Interview
A. Development & Rationale

A structured oral interview was designed to identify candidates with strong interpersonal skills
which our previous job analysis has identified as an essential skill for being a successful clinician
at Shared Vision Psychological Services (SVPS). The job analysis previously done by Team
InnovatI/On breaks down interpersonal skills into the following skills and abilities:
Skill/Ability Definition (Fleishman, 1995)
Active Listening Mindfully and attentively listening to others during a conversation.
Social Being aware and understanding of others' reactions and emotions.

Perceptiveness
Oral Expression The ability to speak fluently and clearly for others to easily understand.
Oral Comprehension The ability to listen and understand spoken words and sentences
Agreeableness The ability to be pleasant, tactful, and helpful when working with others. This ability involves
the degree to which the individual conveys a likeable manner.
Oral Defense The ability to orally defend one's reasoning and conclusions. This ability involves using logic to
defend one's position rather than emotions such as enthusiasm and eagerness.
Negotiation The ability to bargain with others to resolve conflict. This ability involves settling differences
through mutual concession to ensure an acceptable outcome of the parties involved.
Persuasion The ability to present information in order to influence the opinions and actions of others. This
ability involves using persuasive tactics to sway other's thinking or behaviors.
Interviews provide SVPS with a unique opportunity to have interpersonal interactions with job
candidates where they can assess these skills (Levashina, Hartwell, Morgeson, & Campion,
2014). A structured interview provides a standardized approach to organizing, facilitating, and
rating an interview with job candidates. Structured interviews reduce interviewer bias, have
higher inter-rater reliability and test-retest reliability, and provide organizations with more
accurate information about the candidate (Campion, Palmer, & Campion, 1997; Levashina,
Hartwell, Morgeson, & Campion, 2014). This will ultimately lead to better selection decisions,
improving overall employee-organizational fit and job performance of Clinical Psychologists at
SVPS (Levashina, Hartwell, Morgeson, & Campion, 2014).
The format of the structured interview Team InnovatI/On designed follows best practices
suggested by leading industry research (Levashina, Hartwell, Morgeson, & Campion, 2014),
which are listed below:
4
1. Same interview questions for each candidate
2. Limited question prompting
3. 45-60 minute interview length
4. Questions developed through use of job analysis and SME consideration
5. Rubric for rating responses
6. Inclusion of anchored rating scales
7. Minimum three interviewers
B. Interview Format & Questions

● Length: 45-60 Minutes
● Interviewers: 3 Senior Staff Clinicians
● Interview Questions:
1. What are your clinical interests (and please include any work or experience you have
with it and populations you have treated)?
2. Tell me about an assessment instrument in which you feel competent.
3. What are some personal and professional challenges you have faced as a therapist and
how have you addressed them?
4. Tell us about the most difficult client you have interacted with and how did you
approach that relationship?
5. How do you determine or measure client progress?
6. How do you/would you maintain a minor client’s trust while interacting with parents
within a strained family dynamic?
7. What is your experience collaborating with other professionals and/or schools?
C. Rating Scales
Interview responses will be rated by all interviewers on the inclusion of desired content and the
demonstration of interpersonal skills using score sheets. Desired content for these score sheets
was developed by the Subject Matter Experts at SVPS after the creation of the questions.
Interpersonal skills were further defined into specific, observable behaviors for interviewers to
evaluate. Each question score sheet is tailored to each question since different questions are
designed to elicit specific interpersonal skills. Additionally, three overall communication skills
will be scored at the end of the interview including oral comprehension, oral expression, and
active listening. (See Questions & Rating Scales in Appendix A)
5
D. Rating Scale Instructions
1. As interviewers listen to responses from candidate’s, they will “check off” the
interpersonal skills and content presented by candidates on each question score sheet.
2. After the interview is complete, interviewers will complete the score sheets for overall
communication skills.
3. Once all score sheets are filled out, the number of “checked off” boxes for each question
are summed and divided by the total possible score for each question to get a percentage.
4. This percentage will fall into one of three ranges. The range that the percentage falls into
determines the score for each question:
Anchored Rating Scale Scoring Percentage
Exceptional Performance (5) 100-90%
(4) 89-80%
Average Performance (3) 79-70%
(2) 69-60%
Poor Performance (1) 59% and Below
5. Once all 10 scores have been determined, add them up. This sum represents the
candidates structured interview score for each interviewer.
6. Average the scores from all interviewers to determine the candidates final structured
interview score.
E. Rater Training
Senior staff has been trained in-person by Team InnovatI/On to conduct evaluate and score
candidate responses. In-person rater training focused on three main objectives:
1. Orient staff to the interpersonal skills and behavioral definitions that will be used to
evaluate interview responses.
2. Review interview questions and instruct staff on rating scale/scoring procedures for
each question and final candidate scores.
3. Educate staff on common rater errors/biases and how our structured interview is
designed to reduce these.
(See complete Rater Training Packet in Appendix F)
6
F. Validation Process
The structured interview was developed alongside SMEs that work as both incumbents and
supervisors of the clinical psychologist position within SVPS. Several development meetings
were held to determine the appropriate questions to measure the interpersonal skills determined
to be essential for high performance as a clinical psychologist. Once the structured interview
questions were established, the rating criteria was developed by linking to the behavioral
definitions derived from the job description/specifications. Once the linkage process concluded,
a final meeting was held between Team InnovatI/On and SVPS to finalize the structured
interview questions and rating criteria (See Linkage Table in Appendix C).
7
II. Situational Judgement Test (SJT)
A. Development & Rationale
A SJT is being developed in order to assess the approach a job candidate will take when
presented with a description of a clinical case and tasked with determining an interpretation and
clinical treatment plan for that clinical case. A SJT presents candidates with work situations that
they must review and respond to (Ryan, 2014). This will help Shared Vision identify the
candidates who take a psychodynamic approach to the clinical case. Additionally, Shared Vision
will be able to assess the quality of the decisions made from the psychodynamic approach to
determine overall psychodynamic competency.
SJTs have been found to be a valid selection tool, predicting more variance than cognitive ability
tests as well as clinical problem-solving tests. Additionally, SJTs are also the most accepted by
candidates (Koczwara et. al., 2012). While it is important to ensure a valid tool is being used,
candidate acceptability is an important factor to consider as well as this establishes perceived
fairness in the eyes of the candidates.
The SJT development is set to continue in December, 2018 with finalization scheduled for
February 12th, 2019.
IMPORTANT DATES
December 17, 2018 First development meeting will consist of outlining the four vignettes
as well as discuss criteria direction.
January 7, 2019 Second development meeting will review the four vignettes which will
be built upon concluding the December 7th meeting. Additionally,
rating criteria will continue to be discussed and established
January 22, 2019 Third development meeting will review the SJT final draft and finalize
rating criteria.
February 12, 2019 The SJT will be finalized and added to the technical report. In
addition, a training document will be added to establish the proper
procedures for raters.
8
B. Content/Questions
The SJT will come in form of written vignettes. In this case, the situations will involve
individuals in need of treatment through a detailed description of their clinical case. SJTs have
been classified into constructs of knowledge and skills, applied social skills, and basic tendencies
(Ryan, 2014). Additionally, SJTs are used to measure decision making. Matching the construct
measured in the SJT to the nature of performance in the clinical environment will increase the
construct validity (Ryan, 2014). In this case, the psychodynamic theory knowledge and critical
thinking will be reviewed and matched to the item’s construct.
The format of the vignette assessment will follow best practices suggested by industry research
(Evans et. al.,2015), which are listed below:
1. Derived from literature and clinical experience

2. Be clear, well-written, carefully edited
3. Between 50 and 500 words
4. Story-like
5. Similar structure for each vignette
6. Use present tense
Other format options will be explored during the initial development meeting with SMES. The
final product will consist of five vignettes, in which the candidate will have to create a written
response that details how they would approach the situation and determine the appropriate
treatment plan. The test will take place immediately following the structured interview.
Interviewers will leave the room and give the candidate 60 minutes to complete the test.
C. Rating Scale
An anchored rating scale will be used in addition to multiple raters. Similar to the structured
interview, using an anchored rating scale helps standardize the process in order to reduce rater
bias and increase inter-rater and test-retest reliability (Campion, Palmer, & Campion, 1997;
Levashina, Hartwell, Morgeson, & Campion, 2014).
Each vignette response will receive two anchored rating scale scores: one for appropriate clinical
analysis and one for development of a treatment plan using the Psychodynamic approach, for a
total of 10 scores. Anchored rating scale scores are: Exceptional Performance (5), Average
Performance (3), and Poor Performance (1). The highest possible score a candidate can receive is
50 and the lowest is 10.
9
D. Rating Scale Instructions
After the SJT development process is complete, Team InnovatI/On will develop rating scale
instructions and add them to this portion of the Technical Report.
E. Rater Training
When instructions are developed, Team InnovatI/On will train senior staff in-person. Objectives
for that training will be developed and added to this portion of the Technical Report. The rater
training packet will be added to the end of the appendix.
F. Validation Process
Similarly to the structured interview, the job analysis performed by Team InnovatI/On is the base
for identifying the knowledge, skills, abilities, and other characteristics (KSAOs) measured in
the SJT. Team InnovatI/On will work closely with the SVPS subject matter experts (SMEs) to
develop appropriate scenarios to be used for the vignettes. Multiple development meetings have
been scheduled to ensure the content validity of the SJT. Clinical expertise and knowledge will
be critical in the development of this tool, both in creating appropriate scenarios and establishing
dimensions and anchors. Incumbents will be asked to take the SJT and SMEs will then score the
responses and discuss the anchors to ensure the tests validity. As with the structure interview,
content validity will be established for this measure by working closely with the SVPS SMEs.
10
III. Selection Decision
SVPS SMEs will review resumes to determine qualified candidates to interview for the Clinical
Psychologist position. SMEs will be looking for individuals that have completed a clinical
externship, clinical internship, and a Clinical Psychology Psy.D. or Ph.D. program, as well as
experience/knowledge in psychodynamic theory. Once a candidate moves onto the interview
stage, they will also participate in the situational judgement test.
The Structured Interview and Situational Judgement Test will both account for 50% of the
candidate’s total score. This was a topic of great discussion amongst Team InnovatI/On and the
SVPS SMEs. Ultimately, SVPS SMEs determined that the interpersonal communication skills
measured in the structured interview should carry the same weight as the clinical knowledge and
approach measured with the situational judgement test. Additionally, the SMEs felt as though a 5
point likert scale was an appropriate measurement for both the Structured Interview as well as
the Situational Judgement Test. We feel as though “Exceptional”, “Average”, and “Poor” are
appropriate starting points for the anchors, as they should accurately reflect the candidate’s
performance.
Structured Interview Score + Situational Judgement Test Score = Candidate Total Score
(50 Points) (50 Points) 100 Points
Seventy points is the minimum recommended score to be considered for hire in this selection
process. The score of 70-79% is classified as “Average Performance”. Because of this, a score
of 70 is the lowest score a candidate can receive and still be considered qualified to fill the
position of Clinical Psychologist. Once SVPS completes their interviews, candidates will be
rank ordered based on their score. SVPS SMES determined that the highest scoring candidate
will then be considered for the position. In the event of a tie, the SMEs will review the notes to
determine the candidate best fit for the position.
11
IV. Adherence to Legal and Professional Guidelines
Technical Criteria:
Regretfully, we are unable to conduct a validity study for Shared Vision for this selection
procedure as it is not automated, and we do not have those resources. However, our selection
procedure does meet sections 6 and 7 of the Uniform Guidelines.
In order to ensure that the interview process is valid, content- related validity needed to be
established (Uniform Guidelines, 1978). Team InnovatI/On created the content measures based
off of the job description for a clinical psychologist at Shared Vision. In addition, SMEs were
used to establish content validity through various development meetings. Additionally, the
interviewers will be assessing the interviewees through an objective assessment rubric of each
question. Each question has a determined set of criteria that the interviewers will asses for- so,
each interviewee will be measured under the same criteria and have the same expectations.
These are our “Proper safeguards” taken to ensure scores on selection procedures are not
influenced by any outside biases (Uniform Guidelines, 1978). The structure of the interview
focuses on “important or critical work behavior(s) or work outcomes” (Employee Guidelines,
1978). The algorithm for the assessment allows for the “standardization rating of overall work
performance”(Employee Guidelines, 1978).
Alternative Procedures:
Due to the interactive and personal nature of the position of a Clinical Psychologist, it was
necessary to measure the applicants along multiple constructs to guarantee that they have all of
the necessary skills. As part of the skills required to be a successful psychologist are only
observed through interactions, it is imperative that applicants partake in an interview, so the
interviewers can see: their active listening, social perceptiveness, oral expression and oral
comprehension. As a result, an automated hiring component would not be desired for this
interview process.
The structured interview process and situational judgement test allow for a holistic view of the
applicant. It allows for the organization to meet and interact with the interviewee. And, that
interaction is a critical component of being able to assess a clinical psychologists abilities for the
organization.
While the situational judgement tests and some of the questions on the structured interview,
allow for a deeper understanding of the candidates academic skills and past experiences to be
able to apply their knowledge to the job.
12
Adverse Impact:
Adverse impact is the negative effect that an unfair or biased selection process has on a specific
group of people (race, age, gender, etc.) In order to account for adverse impact which is,
“A selection rate for any race, sex, or ethnic group which is less than four-fifths (4/5) (or eighty
percent) of the rate for the group with the highest rate will generally be regarded by the Federal
enforcement agencies as evidence of adverse impact” (Uniform Guidelines, 1978). And, it is
critical for an organization to have detailed records on their selection procedures in order to
avoid unintentional discrimination in the selection process. And, if accurate records are not kept,
“the Federal enforcement agencies may draw an inference of adverse impact of the selection
process from the failure of the user to maintain such data” (Uniform Guidelines, 19798
To prevent adverse impact, Team InnovatI/On has created a structured interview process with
predetermined constructs that the interviewee will be assessed on. In the structured interview,
each question will be measured based upon specific components that have a correlating rubric to
properly message each applicant's response. Therefore, each applicant will be assessed fairly and
accurately as they are being measured through an objective process. Interviewers will also be
trained on how to properly interview and score a candidate with established norms based off of
“Best Practices” (Levashina, Hartwell, Morgeson, & Campion, 2014).
Understandably, there will be fewer applicants as a Clinical Psychologists are a very niche
market. And, as such, the Uniform Guidelines have accounted for this possibility and state that,
“where special recruiting or other programs cause the pool of minority or female candidates to be
atypical of the normal pool of applicants from that group” (Uniform Guidelines, 1978). Hence, if
the hirees tend towards a specific demographic- due to the narrow applicant pool that is an
exception to the hiring process.
Affirmative Action:
Affirmative Action is the policy of promoting education and employment to people otherwise
known to suffer from discrimination. However, affirmative action programs “may in design and
execution be race, color, sex, or ethnic conscious, selection procedures under such programs
should be based upon the ability or relative ability to do the work” (Uniform Guidelines, 1978).
Shared Vision welcomes all applicants to their organization. And, the standardized interview
process allows for the clear and objective assessment of employees for the organization and the
focus is on their “ability or relative ability to do the work” (Uniform Guidelines, 1978).
13
VI. Project Cost
Total estimated hours: 100

Estimated cost per hour: $75.00
Total estimated cost: $7,500.00
Once SVPS employees have been trained on both instruments, all Structured Interview and
Situational Judgement Test materials will be handed over to SVPS. There is no additional cost to
the organization once all materials have been handed over.
14
APPENDIX A
Interview Scoring Chart
INDIVIDUAL RATER ASSESSMENT: STRUCTURED INTERVIEW (50%)
Question Score
Oral Expression
Oral Comprehension
Active Listening
Total ____/ 50 (Half of Interview Assessment)
15
Question 1: What are your clinical interests (and please include any work or experience
you have with it and populations you have treated)?
Discusses understanding of emotions based off of nonverbal cues
Responds with the appropriate emotion to a client’s situation
Responds to social cues by interviewer(s)
Responds professionally to interview questions
Presents as likeable
Demonstrates politeness during conversation
Conclusions are derived from clearly communicated reasoning
Discusses interest in assessments
Discusses interest in psychodynamic theory
Experience with psychodynamic theory
Experience with children
Experience with adolescence
Experience with adults
Experience with geriatrics
Total ___/14 = ____% = Circle Correlating Anchor Score
(5) Exceptional Performance (4) (3) Average (2) (1) Poor

100-90% 89-80% 79-70% 69-60% 59-0%
Additional Notes:
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
16
Question 2: Tell me about an assessment instrument in which you feel competent.
Uses various resources to convey a message
Experience with developmental assessments
Experience with social assessments
Experience with neuropsychological assessments
(5) Exceptional Performance (4) (3) Average (2) (1)

Poor
100-90% 89-80% 79-70% 69-60% 59-0%
Additional Notes:
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
17
Question 3: What are some personal and professional challenges you have faced as a
therapist and how have you addressed them?
Is consistently able to resolve conflict
Prepares solutions for objections raised by client
Is flexible in solutions for client
Stays focused on the objective when communicating with client
Displays tact when discussing previous work conflicts
Describes previous pro-social behavior in the workplace
Displays signs of empathy when presented with a difficult client situation
Utilizes logic to defend their position
Avoids use of emotions when defending their position
Provides sound arguments to support reasoning
Offers counterpoints to strengthen their position
(5) Exceptional Performance (4) (3) Average (2) (1) Pr

100-90% 89-80% 79-70% 69-60% 59-0%
18
Additional Notes:
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
19
Question 4: Tell us about the most difficult client you have interacted with and how did
you approach that relationship?
Demonstrates willingness to negotiate
Utilizes leverage
Negotiation result is similar to original objective
20
100-90% 89-80% 79-70% 69-60% 59-0%
Additional Notes:
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
21
Question 5: How do you determine or measure client progress?
Refers to Psychodynamic measurements of progress

100-90% 89-80% 79-70% 69-60% 59-0%
Additional Notes:
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
22
Question 6: How do you/would you maintain a minor client’s trust while interacting with
parents within a strained family dynamic?
Utilizes leverage
Discusses allowing the client to inform guardians of any important issue(s)
23
100-90% 89-80% 79-70% 69-60% 59-0%
Additional Notes:
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
24
Question 7: What is your experience collaborating with other professionals and/or
schools?
Describes experience collaborating with other clinicians
Describes experience collaborating with other professionals other than clinicians
Discusses the importance of collaboration

100-90% 89-80% 79-70% 69-60% 59-0%
Additional Notes:
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
25
Oral Expression: The ability to speak fluently and clearly for others to easily
understand.
Words are enunciated properly
Speaks in complete sentences
Speaks at an appropriate pace
Uses appropriate transitions when speaking
Speaks at an appropriate volume

100-90% 89-80% 79-70% 69-60% 59-0%
Additional Notes:
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
26
Oral Comprehension: The ability to listen and understand spoken words and sentences
Asks appropriate questions
Gives appropriate feedback
Able to connect presented situation to previous learning
Demonstrates appropriate vocabulary
Provides appropriate responses to questions

100-90% 89-80% 79-70% 69-60% 59-0%
Additional Notes:
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
27
Active Listening: Mindfully and attentively listening to others during a conversation.
Displays body language that suggests they are engaged in listening (e.g. head
nod, upright posture, attentive facial expression)
Maintains appropriate level of eye contact with interviewers
Paraphrases information given by interviewer(s)
Provides brief verbal cues like “I see”, “I understand”, “sure”
Waits for appropriate time to speak, allowing interviewer(s) to finish

communicating their thoughts

100-90% 89-80% 79-70% 69-60% 59-0%
Additional Notes:
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
28
APPENDIX B
Interpersonal Skills: Behavioral Definitions

Social Perceptiveness: Being aware and understanding of others' reactions and emotions.
Negotiation: The ability to bargain with others to resolve conflict. This ability involves settling
differences through mutual concession to ensure an acceptable outcome of the parties involved.
Utilizes leverage
Persuasion: The ability to present information in order to influence the opinions and actions of
others. This ability involves using persuasive tactics to sway other's thinking or behaviors.
Agreeableness:The ability to be pleasant, tactful, and helpful when working with others. This ability
involves the degree to which the individual conveys a likeable manner.
29
Oral Defense: The ability to orally defend one's reasoning and conclusions. This ability involves
using logic to defend one's position rather than emotions such as enthusiasm and eagerness.
Oral Expression: The ability to speak fluently and clearly for others to easily understand.
Displays body language that suggests they are engaged in listening (e.g. head nod, upright posture,
attentive facial expression)
Waits for appropriate time to speak, allowing interviewer(s) to finish communicating their thoughts
30
APPENDIX C
Linkage Table: The above table demonstrates which questions correlate with what content area. For
example, SP (Social Perceptiveness) is measured in all four questions. This table allows the rater to
clearly see how each question is being utilized in the interview to find the best fitting applicant.
31
APPENDIX D
Individual Rater Assessment
Situational Judgement Tests (50%)
Ex. 1: Clinical Analysis
Ex. 1: Psychodynamic Approach
Total ____ / 50
32
APPENDIX E
RATERS AVERAGE ASSESSMENT OF CANDIDATE
Structured Interview Situational Judgement Test
Rater 1 Score Rater 1 Score
Average Score ___/3 = __T1___ Average Score ___/3 = __T2___
T1 + T2 = FINAL INTERVIEW SCORE ________
33
APPENDIX F
Clinical Psychologist Structured Interview Instructions

Overview:
● Length: 45-60 Minutes
● Interviewers: 3 Senior Staff Clinicians
● Goal: Observe & Evaluate Candidate Interpersonal Skills
Interview Questions:
1. What are your clinical interests (and please include any work or experience you have
with it and populations you have treated)?
2. Tell me about an assessment instrument in which you feel competent.
3. What are some personal and professional challenges you have faced as a therapist and
how have you addressed them?
4. Tell us about the most difficult client you have interacted with and how did you approach
that relationship?
5. How do you determine or measure client progress?
6. How do you/would you maintain a minor client’s trust while interacting with parents
within a strained family dynamic?
7. What is your experience collaborating with other professionals and/or schools?
Interpersonal Skills to be Evaluated:

● Social Perceptiveness
● Negotiation
● Persuasion
● Agreeableness
● Oral Defense
● Oral Expression
● Oral Comprehension
How to Observe Interpersonal Skills:

The interpersonal skills listed above have been broken down into observable behaviors. See
tables below for these behaviors:
Interpersonal Skills: Behavioral Definitions

Social Perceptiveness: Being aware and understanding of others' reactions and emotions.
34
Negotiation: The ability to bargain with others to resolve conflict. This ability involves settling
differences through mutual concession to ensure an acceptable outcome of the parties involved.
Utilizes leverage
Persuasion: The ability to present information in order to influence the opinions and actions of
others. This ability involves using persuasive tactics to sway other's thinking or behaviors.
Agreeableness:The ability to be pleasant, tactful, and helpful when working with others. This ability
involves the degree to which the individual conveys a likeable manner.
Oral Defense: The ability to orally defend one's reasoning and conclusions. This ability involves
using logic to defend one's position rather than emotions such as enthusiasm and eagerness.
35
Oral Expression: The ability to speak fluently and clearly for others to easily understand.
Displays body language that suggests they are engaged in listening (e.g. head nod, upright posture,
attentive facial expression)
Waits for appropriate time to speak, allowing interviewer(s) to finish communicating their thoughts
36
Question:
Before the interview, please review the questions that you will be asking during the interview.
Each question, as viewed through the linkage table, correlates with interpersonal skills and
specific criteria that SVPS is looking for in their Clinical Psychologists. Therefore, it is also
crucial that you familiarize yourself with the criteria below each question- so you know what to
think about AND what to listen for during the interview, so you can choose the candidate that is
the best possible fit for your organization.
Scoring:
While listening to the applicant, please score them correctly. When you hear the applicant
discuss or demonstrate a behavior or criteria of the score sheet, please check it off. By checking
it off, you’re indicating that they met this criteria. It is critical that you listen closely and check
off the applicable criteria, so that the interview process is valid and reliable. Please add any
additional notes below the score sheet. See example score sheet below:
Displays body language that suggests they are engaged in listening (e.g. head
nod, upright posture, attentive facial expression)
Waits for appropriate time to speak, allowing interviewer(s) to finish

communicating their thoughts

100-90% 89-80% 79-70% 69-60% 59-0%
Additional Notes:
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
37
Rater Instructions:
1. Once all score sheets are filled out, the number of “checked off” boxes for each question
are summed and divided by the total possible score for each question to get a percentage.
2. This percentage will fall into one of three ranges. The range that the percentage falls into
determines the score for each question:
Anchored Rating Scale Scoring Percentage
Exceptional Performance (5) 100-90%
(4) 89-80%
Average Performance (3) 79-70%
(2) 69-60%
Poor Performance (1) 59% and Below
3. Once all 10 scores have been determined, add them up. This sum represents the
candidates structured interview score for each interviewer.
4. Average the scores from all interviewers to determine the candidates final structured
interview score.
CAUTION! AVOID THESE RATER BIASES

● Halo Effect
● Confirmation Bias
● Anchoring Bias
● Bandwagon Effect
● Hindsight Bias
● Leniency & Severity
38
References
Campion MA, Palmer DK, Campion JE. ( 1997 ). A review of structure in the selection
interview. Personnel Psychology, 50, 655 – 702.
Evans, S., Roberts, M., Keeley, J., Blossom, J., Amaro, C., Garcia, A., OdarStough, C, Canter,
K., Robles, R., Reed, G. (2015). Vignette methodologies for studying clinicians’
decision-making: Validity, utility, and application in ICD-11 field studies. International
Journal of Clinical and Health Psychology, Volume 15, pp. 160-170.
Fleishman, E.A. (1995) Fleishman job analysis survey. Rating scale booklet. Potomac:
Management Research Institute (MRI).
Koczwara, A., Patterson, F., Zibarras, L., Kerrin, M., Irish, B., & Wilkinson, M. (2012).
Evaluating cognitive ability, knowledge tests and situational judgement tests for
postgraduate selection. Medical Education, 46( 4), 399–408.
Levashina, J., Hartwell, C,. J., Morgeson, F. P., & Campion, M. A. (2014). The structured
employment interview: Narrative and quantitative review of the research literature.
Personnel Psychology, 67, 241-293.
Ryan, A. M. & Ployhart, R. E., (2014). A century of selection. Annual Review of Psychology, 65,
693-717.
39
Society for Industrial and Organizational Psychology, Inc. (2003). Principles for the validation
and use of personnel selection procedures (4th ed.). College Park, MD.
Uniform guidelines on employee selection procedures (1979). Federal Register, 43,
38290-38315.
40

Selection Procedure Technical Report: December 10, 2018

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Selection Procedure Technical Report: December 10, 2018

Uploaded by

Copyright:

Available Formats

Selection Procedure Technical Report

Prepared for: Shared Vision Psychological Services

December 10, 2018

SELECTION PROCEDURE COMPONENTS

2. Situational Judgement Test

Component Page Number

Structured Interview Development and Rationale 4

Interview Format and Questions 5

Rating and Validation 6-7

Situational Judgement Test 8

Rating and Validation 9-10

A. Development & Rationale

Skill/Ability Definition ​ ​(Fleishman, 1995)

Active Listening Mindfully and attentively listening to others during a conversation.

Social Being aware and understanding of others' reactions and emotions.

B. Interview Format & Questions

Anchored Rating Scale Scoring Percentage

Exceptional Performance (5) 100-90%

Average Performance (3) 79-70%

Poor Performance (1) 59% and Below

(See complete Rater Training Packet in Appendix F)

A. Development & Rationale

1. Derived from literature and clinical experience

Total estimated hours: ​100

INDIVIDUAL RATER ASSESSMENT: STRUCTURED INTERVIEW (50%)

Total ____/ 50 (Half of Interview Assessment)

Discusses understanding of emotions based off of nonverbal cues

Responds with the appropriate emotion to a client’s situation

Responds to social cues by interviewer(s)

Responds professionally to interview questions

Demonstrates politeness during conversation

Conclusions are derived from clearly communicated reasoning

Discusses interest in assessments

Discusses interest in psychodynamic theory

Experience with psychodynamic theory

Experience with children

Experience with adolescence

Experience with adults

Experience with geriatrics

Total ___/14 = ____% = Circle Correlating Anchor Score

(5) Exceptional Performance (4) (3) Average (2) (1) Poor

Responds professionally to interview questions

Discusses understanding of emotions based off of nonverbal cues

Responds to social cues by interviewer(s)

Responds professionally to interview questions

Uses various resources to convey a message

Demonstrates politeness during conversation

Conclusions are derived from clearly communicated reasoning

Experience with developmental assessments

Experience with social assessments

Experience with neuropsychological assessments

Total ___/11 = ____% = Circle Correlating Anchor Score

(5) Exceptional Performance (4) (3) Average (2) (1)

Discusses understanding of emotions based off of nonverbal cues

Responds with the appropriate emotion to a client’s situation

Responds to social cues by interviewer(s)

Responds professionally to interview questions

Is consistently able to resolve conflict

Prepares solutions for objections raised by client

Is flexible in solutions for client

Stays focused on the objective when communicating with client

Displays tact when discussing previous work conflicts

Describes previous pro-social behavior in the workplace

Skill/Ability Definition (Fleishman, 1995)

Total estimated hours: 100

Total _/14 = __% = Circle Correlating Anchor Score

Total _/11 = __% = Circle Correlating Anchor Score

Total _/19 = __% = Circle Correlating Anchor Score

Total _/18 = __% = Circle Correlating Anchor Score

Total _/13 = __% = Circle Correlating Anchor Score

Total _/19 = __% = Circle Correlating Anchor Score

Total _/8 = __% = Circle Correlating Anchor Score