Professional Documents
Culture Documents
The predictive validity of the team diagnostic survey: Testing a model with
performance and satisfaction as output variables
Per Eisele
Article information:
To cite this document:
Per Eisele , (2015),"The predictive validity of the team diagnostic survey", Team Performance
Management, Vol. 21 Iss 5/6 pp. 293 - 306
Permanent link to this document:
http://dx.doi.org/10.1108/TPM-08-2014-0049
Downloaded on: 05 August 2015, At: 08:02 (PT)
Downloaded by Doctor per eisele At 08:02 05 August 2015 (PT)
Abstract
Downloaded by Doctor per eisele At 08:02 05 August 2015 (PT)
Purpose – The aims of the present study were to test the predictive validity of the Swedish version of
the Team Diagnostic Survey (TDS).
Design/methodology/approach – A model with both performance and satisfaction was tested with
structural equation model (SEM) analyses. Participants completing the survey were employees (N ⫽
214) across three large workplaces. Analyses were done at the group level and data from 33 teams were
included in the final data material.
Findings – Results from validation data indicate that the TDS has satisfactory high Cronbach’s alpha
values on most factors. Results from the SEM analyses show a moderate model fit for the main model.
Team-level factors predict both performance and satisfaction, while organization and coaching factors
do not.
Research limitations/implications – The present study was limited to a cross-sectional design,
but earlier studies have shown that the accuracy of the TDS remains consistent over time. The main
purpose of this study was to test the predictive validity of the instrument. Theoretical implications of
the study are that a survey can be used to get a valid overall picture of the real-life work team’s
effectiveness.
Practical implications – Practical implications of the study are that communication between
researcher and/or consultant and organizational stakeholders is made easier, as the most important
factors that affect team effectiveness are identified.
Originality/value – The work on the development on TDS has shown that it is possible to use a
complex instrument to diagnose work groups, and this line of research is leading the way for better
instruments.
Keywords Team working, Team management, Group work, Group behaviour
Paper type Research paper
The aims of the present study were to further validate the Swedish language version of
the Team Diagnostic Survey (TDS). Earlier studies (Wageman et al., 2005, Eisele, 2013)
have shown that TDS is psychometrically sound and that it has sufficiently high
internal consistency and discriminant validity. The emphasis on the present study was
on the predictive validity of TDS.
There are numerous instruments trying to measure team performance, but very few Team Performance Management
address a sufficient amount of factors. Beside TDS, there is the Aston Team Vol. 21 No. 5/6, 2015
pp. 293-306
Performance Inventory (ATPI) which have been created by Dawson et al. (2006) and © Emerald Group Publishing Limited
1352-7592
recently validated by Callea et al. (2014). DOI 10.1108/TPM-08-2014-0049
TPM The TDS addresses the most prominent factors that are known to affect performance
21,5/6 and reflects Hackman’s (2002) model of team effectiveness. The TDS was created to be
a useful tool for giving advice (Wageman et al., 2005). In a field experiment, it was shown
that the TDS can be fruitful as a feedback tool (Eisele, 2013).
Team effectiveness is an evaluation of the outcomes of team performance processes
relative to some set of criteria (Hackman, 1987). Models of team effectiveness exist at
294 different levels, contextual or generalizable. Some are task-specific frameworks or
models that focus on a specific team process or function (Salas et al., 2008). Hackman’s
(1992) model of team effectiveness (1992) is a generalizable model within the most
common input–process– output (I–P–O) framework.
The basic structure of Hackman’s (1992, 2002) model of group effectiveness identifies
three main factors of work team effectiveness:
(1) a group structure that promotes competent work on the task;
(2) an organizational context that supports and reinforces excellence; and
Downloaded by Doctor per eisele At 08:02 05 August 2015 (PT)
Group structure refers to task clarity, group composition and core norms.
Organizational context includes the organization’s reward, educational and
informational systems. External assistance is a factor that pinpoints that an
organization should provide the teams with adequate resources to learn these skills
(Hackman, 2002). Thus, TDS incorporates most important factors that are essential for
successful teamwork but exclude inter-team interactions.
Real team
The real team conditions have three features. Members and non-members are
distinguished so that the team is bounded. Members have some common purpose and
collective responsibility, making them interdependent. Members in the real team have
stability of membership, making it possible to learn how to work together well. So, real
teams are bounded, interdependent and stable.
Compelling direction
This condition is about how the team’s purposes are specified. Good team direction is
clear, challenging and consequential. Challenging direction energizes members, clear
directions orient them toward their main goal and a consequential direction engages
members’ abilities.
In the TDS framework, goal setting or compelling direction is measured in terms of
clarity, challenge and consequentiality of team purposes.
Alper et al. (1998) investigated effects of two different goals (cooperative and Team
competitive) on efficient decision-making of self-managing teams. The strong positive diagnostic
effect of cooperative goals has implications for team leadership that should encourage survey
groups to create cooperative goals on their own. In a study by Hertel et al. (2004) on
management practices related to goals, tasks and output interdependence correlated
with effectiveness of virtual teams. Aube and Rousseau (2005) found a positive
relationship between team goal commitment and different criteria of team effectiveness, 295
and additionally, a moderating role of task interdependence and a mediating role of
supportive behavior. Rousseau and Aube (2010) found that team self-managing
behaviors was positively related to team effectiveness (team performance, team
viability and team process improvement), with a moderating effect of task routineness.
Passos and Caetano (2005) tested a model of the effects of intragroup conflict on team
performance and affective responses, based on Jehns (1995) distinction between three
different types of conflicts: task, relationship and process. Affective responses were
Downloaded by Doctor per eisele At 08:02 05 August 2015 (PT)
measured via direct assessment of satisfaction with team, team function and
communication among members, team leadership and relationship climate. In the TDS,
individual well-being is measured in a similar way.
Enabling structure
It is well established that task structure that is clear and consistent with a group’s
purpose has positive effects on performance. But the TDS also acknowledges
meaningful shared responsibility and the opportunity to learn how well the team is
doing as factors that are most important for team performance. One of the most common
mistakes organizations make when they start with team work is that the groups are not
set up right. As described by Wageman et al. (2005), sometimes, the team’s structure is
labored and strained which can create obstacles in getting things done and, at other
times, groups are neglected and missed. The latter often because people assume that
group work is so superior to individual work that groups perform better than single
individuals even without any deliberate work on their structure. Additionally,
sometimes the direct observable structure is faulty. The model that describes the
foundation of TDS describes three structural features: task design, team composition
and norms of conduct.
Edwards et al. (2006) examined how team composition based on members’ abilities is
related to team mental models and how these two constructs affect team performance.
The strong direct relationship between composition and performance highlight the
importance of this factor.
Diversity affects team effectiveness differently depending on circumstances. Put
simply, a high diversity is associated with potentially higher creativity but only if
coordination problems can be solved. Whether group members view the team as
sufficiently diverse is dependent not only on the degree of diversity per se but also on
how the diversity is perceived. As an example of current research on team diversity,
Peters and Karren (2009) investigated the relationship between trust, diversity and team
performance, and trust was found to mediate the relationship between diversity and
team performance. Stewart and Johnson (2009) found that, in more gender-diverse
groups leader–member exchange (LMX) differentiation was positively associated with
work group performance.
TPM Team norms are the shared beliefs regarding the social behaviors expected of its
21,5/6 members. Taggar and Ellis (2007) developed and assessed a model that depicts the
impact of team leaders and staff on negotiated and agreed upon team norms for
collaborative problem-solving in newly formed teams. Expectations had a significant
impact on the norms that teams adopted around collaborative problem-solving. Norms
about how to solve problems in teams significantly influenced individual team
296 members’ problem-solving behaviors. Earlier studies examining the impact of norms on
group outcomes also show that beneficial norms affect team performance (Feldman,
1984). There have been a few recent studies providing insight into norm formation,
including the influence of demographic heterogeneity on the emergence of cooperative
norms (Chatman and Flynn, 2001) and the development of strategic norms (Levine et al.,
2000). Although it is clear that team norms guide performance relevant behavior, there
is a need for studies in field settings.
Downloaded by Doctor per eisele At 08:02 05 August 2015 (PT)
Team leadership
In studying five leadership behavior factors (aversive, directive, transactional,
transformational and empowering), it was concluded that both vertical and shared
leadership was related to team effectiveness (Pearce and Sims, 2002). In the TDS
framework, team leadership is connected with the identified factors that affect team
effectiveness and how the leader acts toward those factors.
The TDS attempts to measure four different kinds of coaching: task-focused, operant,
Downloaded by Doctor per eisele At 08:02 05 August 2015 (PT)
interpersonal and unhelpful directives, all regarding both team leader coaching and peer
coaching. Task-focused coaching is about building commitment. Operant coaching is
about giving appropriate feedback. Interpersonal coaching is equivalent to helping
behavior and can, for example, be about conflicts. Examples of unhelpful directives are
over-instruction or fault-detection.
Team coaching theory (Hackman and Wageman, 2005) posits that team effectiveness
is a joint function of three performance processes:
(1) Choice of performance strategies is influenced by core norms and the
organizational information system.
(2) The amount of effort that is influenced by the task design and by the
organization reward system.
(3) The amount of knowledge and skill members contributes to the team that is
influenced by team composition.
Coaching can be motivational, consultative and educational and both availability and
helpfulness should be measured.
Liu et al. (2009) showed that team coaching had positive effects on team performance
processes regarding effort, skill and knowledge, which in turn had a direct impact on
team effectiveness.
Leader coaching behavior has been found to affect performance (Wageman, 2001) but
coaching behavior of peers seems to be a neglected research field.
Coaching functions are those interventions that inhibit process losses and foster
process gains; motivational coaching addresses effort; consultative coaching addresses
performance strategy; and educational coaching addresses distribution of knowledge
and skill among group members (Hackman and Wageman, 2005).
Method
Participants
Data from 33 teams across 8 different workplaces were collected. The workplaces were
large companies from the state sector. All teams at each workplace were selected for the
data collection. Participants (N ⫽ 214) belonged to three types of work teams:
performing teams (n ⫽ 33, 8 teams), production teams (n ⫽ 103, 21 teams) and service
teams (n ⫽ 21, 4 teams) across four different types of work organizations. In total,
72 group members come from white-collar organizations, 66 group members from
blue-collar organizations, 21 group members from service organizations and 45 group
members from work organizations dealing with education.
The response rate within each company was 90 per cent, and the response rate within
each group was 88 per cent.
Material
The TDS questionnaire
For this study, a new shorter version of TDS was used. The reason for using a shorter
version was based on improvement of internal consistency and more manageable size of
the questionnaire. Creating shorter versions of the TDS is a task of several research
groups around the world. Except where otherwise noted, all items make use of a
five-point scale ranging from highly inaccurate (1) to highly accurate (5). Group-level
composite scores are computed by averaging across items and respondents.
Reverse-scored items are computed before analysis.
Demographic information (five items) includes, for example: “How long have you
been a member of the team you described in this survey?”
Statements about the team includes, for example: “Both the purposes of our team and
the means or purposes we are suggested to use in our work are specified in detail by
others”; and “Our team has also the authority to specify what our team exists to
accomplish, its main purposes”.
Real team (six items) includes, for example: “My work on this team is just one part of
my overall job in this organization”; and “The purposes of this team don’t make much of
a difference to anybody else”.
Compelling direction (five items) includes, for example: “This team has not broad
enough experience to manage goals”.
Enabling structure includes team composition (seven items), task design (seven Team
items) and group norms (three items) – in total, 18 items on a five-point Likert-scale. diagnostic
Example items: “This team has too few members for what it has to accomplish”; and
“Team members have enough skills to leave up to expectations”.
survey
Organizational context (ten items) includes, for example: “Even teams that do an
especially good job are not recognized by the organization”.
Team coaching (four items) includes, for example: “Experts on team facilitation are 299
available”.
Team leadership (13 items) includes, for example: “The team leader helps the team to
develop a shared involvement for goals and purposes”.
Team members’ behavior (five items) includes, for example: “Regular team members take
initiatives to constructively resolve any problems or conflicts that develop among
members”.
Process criteria (nine items). The scales assess the level of effort members collectively
Downloaded by Doctor per eisele At 08:02 05 August 2015 (PT)
expend on the task, the quality of team task performance strategies and the degree to
which the team uses members’ knowledge and skill. Example items: “Members
demonstrate their commitment to our team by putting in extra time and effort to help it
succeed”, “Our team often falls into mindless routines, without noticing any changes
that may have been offered in our situation” and “Members of our team actively share
their special knowledge and expertise with one another.”
Interpersonal processes and individual well-being (19 items). Quality of team
interaction, example item: “Working together energizes and uplifts members of our
team”. Satisfaction with team relationship, example item: “My relations with other team
members are strained”. (R) Satisfaction growth, example item: “My own creativity and
initiative are suppressed by this team”. (R) General satisfaction, example item: “I enjoy
the kind of work we do in this team”. Internal work motivation, example item: “I feel bad
and unhappy when our team has performed poorly”.
Performance evaluation
To perform well groups must:
• make an effort;
• make use of knowledge, abilities and skills of group members; and
• choose appropriate task performance strategies.
300 The scores were conducted by one selected group member from each group, a team
leader and one independent observer. All scores were transformed to a five-degree scale
after all data being collected.
Procedure
The original Swedish version of TDS was translated and back-translated and its
validity was tested in an earlier study (Eisele, 2013). Further validation analyses of the
shorter version used in the present study was conducted prior to this data collection. The
new shorter version has increased internal consistency and discriminant validity.
Downloaded by Doctor per eisele At 08:02 05 August 2015 (PT)
The questionnaire was distributed to work teams from different companies and was
sent to group members by mail and data was saved in an SPSS file after completion of
the survey. Each participant wrote down a name for their group, agreed upon before
data collection. Also, each participant wrote down a name in the demographic section
of the Web survey but was informed that they could fill out the questionnaire
anonymously. Those who wished to remain unknown wrote down a fictitious name.
This individual name and the name of each group made it possible to identify each
respondent with a maintained high degree of confidentiality.
Performance was evaluated by team members, team leaders and one independent
observer.
Result
Psychometric information on the TDS scales for 33 teams are presented for the
5 enabling conditions, the organizational support variables, the coaching variables and
the performance criterion variables in Tables I and II. Descriptive statistics include
means, standard deviations, average correlation within each scale, average correlation
between the scales and internal consistency reliabilities. The latter presented both at
individual level and group level; otherwise, all analyses were made on the group level.
As Table I shows, the discriminant validities are, in general, acceptable but
unsatisfactory for consequentiality, size and organizational resources. Most factors
presented in Table I have satisfactory Cronbach’s alpha values but note two factors,
consequentiality and organizational resources. All factors presented in Table II have
satisfactory Cronbach’s alpha values.
Table II presents psychometric data for the coaching measures, criterion measures
and output variables. These factors had in general satisfactory validity and reliability.
find a suitable model. SEM also allows multiple measures to be associated with a single
latent construct. The decision to make use of SEM was largely a result of discussions on
development of TDS at international conferences.
Goodness-of-fit was calculated to establish whether the overall model was
acceptable. Confirmatory goodness-of-fit analyses of TDS measures and output
variables where done with Amos statistics. Several statistical tests were submitted to
test the adequacy of the proposed model. Chi-square dived with degrees of freedom
(CMIN/DF), root mean square error of approximation (RMSEA) and comparative fit
index (CFI). There was a moderate model fit for the three main models (Table III).
Because the model was acceptable, further analyses were made to establish whether
specific paths were significant. Team factors had stronger regression weights than both
organizational factors and factors related to coaching. Structural factors stronger
impact than for example compelling direction. The latent structure effectiveness load on
both process criteria and satisfaction (Figure 1).
TPM Team effectivness
21,5/6 conditions Items M SD R within R between Alpha individual Alpha team
Process criteria
Effort 3 3.46 0.99 0.41 0.40 0.64 0.78
Performance strategy 3 3.52 0.85 0.49 0.44 0.56 0.72
302 Knowledge and skill 3 3.66 0.85 0.56 0.46 0.73 0.77
Leader coaching
Task-focused 6 3.42 0.93 0.83 0.67 0.86 0.98
Operant 2 3.55 0.57 0.98 0.69 0.87 0.96
Interpersonal 2 3.06 0.99 0.88 0.80 0.86 0.96
Unhelpful directives 3 3.09 0.61 0.86 0.74 0.84 0.94
Peer coaching
Task-focused 3 3.27 0.84 0.96 0.89 0.71 0.98
Downloaded by Doctor per eisele At 08:02 05 August 2015 (PT)
Discussion
This study was the first attempt to examine the predictive validity of the TDS. In
doing so, Hackman’s (2002) model of team effectiveness was tested. SEM was used
due to the complexity of the subject area and the confirmatory nature of the
proposed model.
The models were admissible and gave several insights. The most important insight
being that the internal team factors predicted both performance and satisfaction more
than organizational support and coaching availability.
Team
diagnostic
survey
303
Downloaded by Doctor per eisele At 08:02 05 August 2015 (PT)
Figure 1.
The structural
equations of
Hackman’s (2002)
team effectiveness
model
A limitation of the study was the cross-sectional design. Because team behavior changes
over time (Gutwin and Greenberg, 2004), studies with longitudinal designs should be
used more. However, TDS data have been collected over time in an earlier study (Eisele,
2013).
No significant differences were found between the three types of teams. Thus,
differences in specific task were not further explored in this study and can be seen as
another limitation.
The TDS is a valid tool both for research and consultative work. However, it should
be complemented with investigation into certain specific behaviors such as team plans.
For example, DeChurch and Haas (2008) found that team effectiveness is determined
most strongly by reactive adjustment (occurs during the action), then by contingency
TPM planning (specifies backup plans) and least so by deliberate planning (specifies a
21,5/6 primary course of action).
Theoretical implications of the study are that a survey can be used to get a valid
overall picture of the real life work team’s effectiveness. Although other forms of data
collection should be used also, a reliable questionnaire of team effectiveness is fruitful
for future research on team effectiveness. The addition of an observation procedure was
304 a methodological improvement but did not affect the model per se.
Managerial implications of the study are that communication between researcher
and/or consultant and organizational stakeholders is made easier, as the most important
factors that affect team effectiveness are identified.
The TDS represents a bold attempt to include all of the most important factors that
affect team effectivity, and not just a few easy-to-measure factors. It is a false
assumption that the instrument is more valid if it is limited to a too small number of
factors. The non-measured factors simply make the instrument non-valid. By including
Downloaded by Doctor per eisele At 08:02 05 August 2015 (PT)
the factors with a bit lower reliably the instrument as a whole is in fact more valid. By
excluding them, we only create an illusion of a valid instrument. The work on the
development on TDS has shown that it is possible to use a complex instrument to
diagnose work groups, and this line of research is leading the way for better
instruments. This is even truer when it comes to applied use; data based on TDS
improve communication between researcher/consultant and stakeholders.
To conclude, the new shortened Swedish version of the TDS has high internal
consistency, discriminant validity and predictive validity. The research has increased
understanding of team effectiveness. Knowledge gained by the TDS can be used to
make better interventions. The research suggests that the TDS can facilitate the
planning of group processes, for example, acceptance of all ideas. By collecting
self-reported data based on group members’ own view of group processes and the main
factors that affect these processes, it is possible to proclaim aspects of teamwork that are
not easily observed. The main point is that the type of knowledge given by the TDS can
be used to improve communication between facilitator and stakeholder.
References
Alper, S., Tjosvold, D. and Law, K.S. (1998), “Interdependence and controversy in group decision
making: antecedents to effective self-managing teams”, Organizational Behavior and
Human Decision Processes, Vol. 74 No. 1, pp. 33-52.
Aubé, C. and Rousseau, V. (2005), “Team goal commitment and team effectiveness: the role of task
interdependence and supportive behaviors”, Group Dynamics: Theory, Research, and
Practice, Vol. 9 No. 1, pp. 189-204.
Callea, A., Urbini, F., Benevene, P., Cortini, Di Lemma, M. and West, M.A. (2014), “Psychometric
properties and factor structure of the Italian version of the Aston team performance
inventory”, Team Performance Management, Vol. 20 Nos 1/2, pp. 6-18.
Chatman, J.A. and Flynn, F.J. (2001), “The influence of demographic heterogeneity on the
emergence and consequences of cooperative norms in work teams”, Academy of
Management Review, Vol. 44 No. 5, pp. 956-974.
Dawson, J.F., West, M.A. and Markiewicz, L. (2006), Aston Team Performance Inventory:
Management Set, ASE, London.
DeChurch, L.A. and Haas, C.D. (2008), “Examining team planning through an episodic lens effects Team
of deliberate, contingency, and reactive planning on team effectiveness”, Small Group
Research, Vol. 39 No. 5, pp. 542-568.
diagnostic
Edwards, B.D., Day, E.A., Winfred, A. Jr and Bell, S.T. (2006), “Relationship among team ability
survey
composition, team mental models, and team performance”, Journal of Applied Psychology,
Vol. 91 No. 3, pp. 727-736.
Eisele, P. (2013), “Validation of the team diagnostic survey and a field experiment examine effects 305
of an intervention to increase team effectiveness”, Group Facilitation: A Research and
Applications Journal, Vol. 13 No. 1, pp. 34-49.
Feldman, D.C. (1984), “The development and enforcement of group norms”, Academy of
Management Review, Vol. 9 No. 1, pp. 47-53.
Gutwin, C. and Greenberg, S. (2004), “The importance of awareness for team cognition in
distributed collaboration”, in Salas, E. and Fiore, S.M. (Eds), Team Cognition:
Understanding the Factors that Drive Process and Performance, American Psychological
Downloaded by Doctor per eisele At 08:02 05 August 2015 (PT)
Further reading
Eisele, P. and Arthur, W. Jr (2014), “General performance measure: Further validation of the GPM
in a longitudinal study across different jobs”, Preparation for Scandinavian Journal of
Management.
Hackman, J.R. (1990), Groups that Work (and those that Don’t), Jossey-Bass, San Francisco.
Hackman, J.R. and Morris, C.G. (1975), “Group tasks, group interaction process, and group
performance effectiveness: a review and proposed integration”, in Berkowitz, L. (Ed.),
Advances in Experimental Social Psychology, Academic press, San Diego, CA, pp. 45-99.
Corresponding author
Per Eisele can be contacted at: per.eisele@gmail.com
For instructions on how to order reprints of this article, please visit our website:
www.emeraldgrouppublishing.com/licensing/reprints.htm
Or contact us for further details: permissions@emeraldinsight.com