Wer 53,2016) 09-619 o DORN S2F3OR S257 TOs Cross-cultural adaptation of the Individual Work Performance Questionnaire Linda Koopmans", Claire M. Bernaards*, Vincent H. Hildebrandt", Debra Leer’, Henrica C.W, de Vet and Allard J. van der Beek*** "Body @ Work, Research Center for Physical Activity, Work and Health, TNO-VU University Medical Center, Amsterdam, The Netherlands “Expertise Center Life Style, TNO, Leiden, The Netherlands, “The Program on Health, Work and Productivity, Institute for Clinical Research and Health Policy Studies, Tufis Medical Cemter, Boston, MA, USA “Department of Epidemiology and Biostatistics, EMGO+ Institute for Health and Care Research, VU University Medical Center, Amsterdam, The Netherlands "Department of Public and Occupational Health, EMGO*+ Institute for Health and Care Research, VU University Medical Center, Amsterdam, The Netherlands Received 7 March 2014 Accepted 18 Febrary 2015 Abstract. BACKGROUND: The Individual Work Performance Questionnaire (IWPQ). measuring task performance, contextual per formance, and counterproductive work behavior, was developed in The Netherlands. OBJECTIVES: To cross-culturaly adapt the TWPQ from the Dutch to the American English language, and assess the {questionnair’s internal consistency and content validity inthe American-English context. METHODS: A five stage translation and adaptation process was used: forward translation, synthesis, back-translation, expert committee review, and pilot-testing. During the pilt-esting. cognitive interviews with 40 American workers were performed, to examine the comprehensibility, applicability, and completeness ofthe American-English TWPQ. RESULTS: Questionnaire instructions were lightly modified to id interpretation inthe American-English language. Incon- sistencies with verb tense were identified, and it was decided to consistently use simple past tense. The wording of ive terns ‘was modified to better suit the American-English language. In general, participants were positive on the comprehensbility, applicability and completeness ofthe questionnaire during the pilot-esting phase. Furthermore, the study showed positive results concerning the internal consistency (Cronbach's alphas forthe scales between 0.79-)89) and content validity of the ‘American-English TWPQ. CONCLUSION: The results indicate that the cross-cultural adaptation ofthe American-English IWPQ was successful and thatthe measurement properties of the translated version are promising, Keywords: Job performance, scale, psychometres, translation, validation 1. tneodetion tn day's word it is inreasingy important ‘Adie for corespondence: Aland J. van dee Beck, Depa: maintain, improve, and optimize individual work per: MeteccRacevUtun acre ane Romance of employees. In Europe andthe United ‘terdam, The Netherlands, Tel: +31 20 444 9649; E-mai ‘States of America, for example, the “grey wave" (i.¢., Saboneeeel aocrlerted gro of the oder working population 1051-9815/16/$35.00 © 2016 —10S Press and the authors. All righ reserve610 1. Koopmans et a. / Cross-cultural adaptation of he Individual Work Performance Questionnaire and a decline in numbers of the younger working population) and the economic recession influence companies and employees to perform more or bet~ tet work with less people. Due to the grey wave, the retirement age of older workers has been prolonged [1]. Thus, performance at work has to be maintained until a later age. In order to accurately establish the effectiveness of interventions, procedures and strate- gies to maintain, improve, or optimize individual ‘work performance, valid measurement of individual ‘work performance isa prerequisite Individual work performance, defined as “behar- iors or actions that are relevant to the goals of the organization”, is since long considered to be a ‘multidimensional construct [2. 3]. Based on several reviews of the literature [4-6], it can be concluded that individual work performance consists of three broad dimensions: task performance, contextual per- formance, and counterproduetive work behavior. The first dimension, task performance, traditionally has received most attention, and can be defined as “the proficiency with which individuals perform the core substantive or technical tasks central to theit job” [2]. The second dimension of individual work performance is contextual performance, defined. as “behaviors that support the organizational, social and psychological environment in which the technical core must function” [7]. The third dimension of indi- vidual work performance is counterproductive work behavior. defined as “behavior that harms the well- being of the organization” [5]. While there is a general consensus on the three dimensions of individual work performance [4, 5], there is still litle consensus on how to measure the construct. A multitude of measurement instru- ‘ments to measure individual work performance (or related constructs such as. presenteeism or pro- ductivity) exist [8]. For example, Williams and Anderson [9] developed a short and generic task performance scale, which measured behaviors such as adequately completing assigned duties, fulfil ing prescribed responsibilities, and performing tasks that are expected of the employee. Scales used to assess contextual performance are those devel- coped by for example Podsakoff and MacKenzie [10] for Van Scotter and Motowidlo [11]. The former focuses on measuring altruism, conscientiousness, sportsmanship, courtesy, and civic virtue. The latter focuses on measuring interpersonal facilitation and {ib dedication, Scales used to assess counterproduc- tive work behavior were developed by for example Bennett and Robinson [12] or Spector et al. [13]. ‘The former authors focus on measuring organisa- tional and interpersonal deviance. The latte authors focus on measuring sabotage (eg. damaging com- pany equipment), withdrawal (e.g. taking longer breaks), production deviance (e.g. doing work incor- rectly), theft (eg. stealing company property), and abuse (eg. making fun of someone at work). Tn the field of occupational health, the main focus for mea- suring individual work performance was on sickness absenteeism or presentevism (i., work absence or loses in individual work performance due to health impairments) In accordance, numerous instruments have been developed to measure sickness absen- tecismor presentecism, suchas the Work Productivity ‘And Impairment Questionnaire (14), Work Limita- tions Questionnaite [15], and the WHO Health and Performance Questionnaire [16]. Several limitations can be observed in the scales developed to measure dimensions of individual work performance. Mos strikingly, none of them measure allof the relevant dimensions of individual work per- formance together. Thus, they do not measure the full range of individual work performance. Also, scales mescuring different dimensions ean include antithetical items, creating unjust overlap between these scales [17]. As a result, the content validity of these scales can be questioned. Furthermore, none of the scales appear suitable for genetic use, The scales were developed for specific populations, for example to ascertain the specifi influence of physi- cal and mental health on individual work performance [14-16], or they were developed and refined based on éemployces witha specitie occupation [9, 10}. In order to overcome limitations of existing questionnaires, the Individual Work Performance Questionnaire (IWPQ) [18, 19] was developed in ‘The Netherlands. The IWPQ measures individual \work performance atthe group-level based on indi- vidual worker self-report. ts conceptual framework was derived froma systematic review ofthe literature {6}, and the item poo! was developed from the liter lure, existing questionnaires, and expert interviews, ‘The final items were determined using expert con- sensus [8] and Rasch analysis [18, 19]. The IWPQ is the first questionnaire to incorporate all relevant dimensions of individual work performance into one questionnaire. An advantage of this i thatthe con- tent ofeach scale is complementary tothe content of the other scales. Asa result, the scales do not include redundant items, thats, tems overlapping in content {17 The psychometric properties ofthe IWPQ have been tested in The Netherlands, and indicate good to1. Koopmans et a. / Cross-cultural adaptation of he Individual Work Performance Questionnaire ou excellent internal consisteney for task performance (=0.78), contextual performance (@=0.85) and counterproductive work behavior (w=0.79). Also, the IPQ has shown good face and structural validity [8, 18, 19}, as well as sufficient convergent validity and good discriminative validity {20}, Another important characteristic of the TWPQ is that itis generically applicable [18, 19]. Thus, it is suitable for workers in all types of jobs, and pro- Vides a relatively comprehensive measure of work performance that can capture the potential influence of all sorts of variables (e.g., personal and environ ‘mental variables). Thus, conceivably, the IWPQ may be suitable for examining the effectiveness of a broad range of interventions, procedures and strategies to maintain, improve, or optimize individual work per- formance. In onler for the IWPQ to be used outside of The Netherlands, it has to be eross-culturally adapted and. validated. The objectives of the current study were to perform a cross-cultural adaptation of the IWPQ from the Dutch to the American-English language, and assess the questionnaire’s internal consistency ‘and content validity inthe American-English context, Methods, 21. Individual Work Performance Questionnaire The Individual Work Performance Questionnaire (WPQ) [18, 19] measures “employee behaviors or actions that are relevant to the goals of the organi- zation’ [2]. The IWPQ consists of 18 items, divided into three scales: task performance, contextual per formance, and counterproductive work behavior (see Table 1). All items have a recall period of 3 months and a 5-point rating scale (“seldom” to “alway' task and contextual performance, “never” 1" for counterproductive work behavior). A mean score for each IWPQ scale can be calculated by adding the item seores, and dividing their sum by the number of| items in the scale, Hence, the IWPQ yields three scale scores that range between 0 and 4, with higher scores reflecting higher task and contextual performance, and higher counterproductive work behavior. 2.2. Cross-cultural adaptation The DWPQ's cross-cultural adaptation pro- cess (Fig. 1) followed the stages proposed by Beaton et al [21], and is based on the first step of the three-step process adopted by the International Society for Quality of Life Assessment (IQOLA) project (22) 2.2.1, Forward translation The forward translation of the TWPQ's instruc tion, items, and answer categories, was performed by two independent translators. Both translators were bilingual, with American-English as their mother tongue. One translator (the “informed” translator, researcher) had expertise on individual work per- formance, and the other translator (the “uniformed” translator, a chiropractor) was naive about the topic: Both translators wrote a report of the translation, con- taining challenging phrases and uncertainties, and considerations for their decisions. 2.2.2. Synthesis The results of both translations (TL and 72) ‘were compared by the two translators and one researcher (LK). A written report documented the consensus process, the discrepancies, and how the discrepancies were resolved. The translators and the researcher reached consensus on one common American-English questionnaire (T-12). 2.2.3, Back translation ‘The common American-English questionnaire was back-translated into Dutch by two other indepen- dent translators. Both wanslators were bilingual, ‘with Dutch as their mother tongue, One transla- tor was naive about the topic (a PhD student), ‘whereas the other translator had expertise on the topic (a researcher), Both translators wrote a report of the translation, containing challenging phrases and ‘uncertainties, and considerations for their decisions. 2.24. Expert committee review Al the translated versions were combined into fone pre final questionnaire by an expert commit tee, The expert committee consisted of the four translators, one researcher (LK), and one method- ologist (HAV), Discrepancies between the original and translated versions were identified and dis- cussed. Also, semantic, idiomatic, experiential and conceptual equivalences were evaluated. Again, a ‘written report documented the consensus process, the discrepancies, and how the discrepancies were resolved, The expert committee reached consen- sus on a pre-final American-English version of the 1wPQ,612 Table | ‘The pre-final American-Englsh IWPQ an 1. Koopmans et a. / Cross-cultural adaptation of he Individual Work Performance Questionnaire 1 descriptive statistics af the tems ‘The folonng quetons eset ow you a ou your work rn he at 3 mons nero et an scutes of our const or. is tpt yo complet the questonae x cael al ant yas pose you are uncerain abt how answer ‘pric qucton plese pete bes pile newer The quesonsir wake hs Sno comps. The qcionse fermi apni: you answers wil tbe Sub your supers) cl Respons a (seidom) _Gometines) gua) _foen)_(anay)__(SD)_somelaton Tsk peormance (yale Inte pst nots. Tl Twwahletoplinmy werksotatt 01) 77S) 925) 18GRS) HATS) 270CT) ae ston ine” TH Thainmindtheworkreuik Tanda (0) «2S)— TITS) 12400) 201800) 328asH) ——as8 wach TH Iwaahletodiingushmsinineer 010) «L2S)—TATS) 200500) 120000) 3.081075) 06 rom side ses TM tyanahletocamycutmy work well 125) 10250) 5375) 10S) 44100) DISKO ass with inal ea lon TPS phoned ny work optimal bw 502 WAS) HED -9A25 27909 — OM Contesta performance CP) ele Ine pst 3 mons. CPs nm cine. tsanedaow (0) S125) 8D) WASH) TERS) 9B) 028 sks when my ld ss were C7 ses ncaegng tamer sine 105) 2) SRS) MES) 187) a8) 04 nose ode” Cr workedomeping my woksKils 1) FS) NERS) HATS) BERS 2aS09H) O42 PIO cameupuihcrotvesohtions foe «125-35 ARS) IBS) TITS) ARH) 052 CPI took omexra responses 125 26m GUS) 23675) SDH A8KOSH — OM C2 omy soap ew changin 010) 40) HSH) ASGTS) aT) 2884095) 070 my wok (CPI actly piped in mings 9 saa S428) 15G7S) 16440 308097) 04s “anor conlas, Cawnerprductive work bebnior (CWI) scale Inept nos. CWBIE Templaingaoutuninporamtses 4100) 206500) 13025) 35) OW 38TH — 0D CWDIS Ime problem at workiggerdun —1SG7S) 206500) 4.100) 0 o76a7H 07 CwWHIG TeewadoneseEavengecsefa — 1050) UTS) HAS) AS) Basa) 0D ‘tino work ea CWHIT hed iollegucssbotthenepte $200) 84S) BONO S42 kaa ost pects ny ork” COwDIE lke people ide of he wes IRS — TUT) 4400) Lavan 076 epi aeat neatine specs of my wk” *Dificult to wanslate: Adjusted afc piles. 2.25, Pilot-test ‘To examine the comprehensbilty, applicability, and completeness of the translated questionnaire, a pilottest was performed. A total of 40 partici pants were included in the pilot-test, Inclusion criteria ‘were: currently working (8 hours a week or more). aged 18-65 years, and able to read and understand the American-English language. Participants were a convenience sample of employees of Tufts Medical Center in Boston, MA. In order to promote participa- tion in the pilot-test, an outreach e-mail was send to employees of participating departments, after which an appointment with the researcher (LK) could be ‘made. The pilot-test was approved by the Tufts Uni-1. Koopmans et a. / Cross-cultural adaptation of he Individual Work Performance Questionnaire 613 ‘Stage 1 Forward wanstaon | writen epotof Two tarsatons (Tran) | Sh wanton + Ino age langage “alone & ered trnaitor ‘Stage 2 Syberia Wien report of| = Syesze TH and T2 questionaire T-12) omens proces Wrien report of cach asition “saemonsonb pase ery 01 Sap} ome ina pur sao. Ka soar uous J stad pe USNS + a5 ‘Stage 4 Expert commie review Writen report of "Tamltr, develope, and | consensis process edhadoogit review all reports Reach consensus on pre-inl questionaire Stages: Plott ne 3040 ‘Complete questionaire Probe to get understiding of ‘Weiten epost of urea! emer Fig. 1, Recommended stages of he cross-cultural adaptation pro= ces, based on Beaton etl [21 versity/Tufts Health Sciences Institutional Review Board (IRB number 10929). Afier signing an informed consent form, partici- pants filled in the America-English IWPQ. “Think aloud” and “probing” techniques [23] were used in order to identify participants’ opinion on the comprehensibility, applicability, and completeness of the instructions, items, and answer categories of the translated questionnaire, The duration of the pilot-test was on average 15 minutes, including ques- tionnaire completion. Participants’ comments were written down into a report by the researcher (LK). ‘The comments were independently assessed by wo researchers (LK and CB), after which a consensus ‘meeting took place, Any discrepancies that remained ‘were discussed with the wanslators and the other IWPQ developers (VH, HAY, and AvdB), after which consensus was reached on a final American-Engl questionnaire. 2.3, Measurement properties of the pre-final questionnaire Descriptive statistics of the IWPQ items and scales, and of the socio-demoeraphic characteristies of the participants (gender, age, number of work hours a week, and primary type of occupation) ‘were used to examine the distribution of the IWPQ responses. Internal consistency of the IWPQ scales ‘was determined using Cronbach's alpha [24]. ltem- to-scale correlations were calculated to evaluate the fit of the item within the seale. Furthermore, scale scores were examined for floor or ceiling clfects (>15% at the extreme values [23)). Statistical analyses of the data were done in SPSS20. The content validity of the American-English questionnaire was evaluated by the members oF the expert committee throughout the eross-cultural adap- tation process, and by the developers of the IWPQ through qualitative analysis of the comments pro- ‘vided by the participants ofthe pilot-test 3. Results Bil. Cross-cultural adaptation B.L1, Translation ‘The forward translation of the IWPQ war ‘conducted and some challenging issues were encoun- tered. All issues were discussed among the two trans- lators and the researcher, until consensus emerged. First, conceptual issues were identified with the instruction. “Behavior at work” was considered 100 evaluative and derogatory (implying a lack of matu- rity). To obtain conceptual equivalence to the original ‘meaning, it was chosen to use “how you conducted yourself at work.” Second, for some questionnaire items, inconsistencies with the verb tense were identi- fied. In Dutch, the simple past (e.g.. “started” and the present perfect (e.g., “have started”) are used inter- cchangeably. It was chosen to consistently use the sim- ple past in the American-English version, because the items refer toa completed action in the past 3 months, Furthermore, there were some idiomatic issues in the translation of items TPA, 3 and 4 (“I was able to" versus “I succeeded in”), TP2 (“keep in mind” versus “keep insight"), CP8 and 9 (“keep up-to-date” versus maintain”), CWB14 (“issues” versus “things”) and CWBI6-18 (“aspects” versus sides”). The main rea- sons for choosing the first option for each item were similarity to the original Dutch item, generie applica- bility (suitable for workers in all types of jobs), and appropriateness of a word (decent, proper. ‘The back-translation was conducted without major difficulties. Issues were discussed among the mem- bers of the expert committee until consensusou emerged. First, conceptual issue was identified with the instruction sentence “how you conducted yourself at work.” Comment was that you cannot “conduct yourself." To obtain conceptual equivalence fo the original meaning, it was chosen to use “how you carried out your work.” Second, there were some lin- ‘euistic and conceptual issues in the wording of items ‘TP2 (“results I needed to achieve in my work” was considered incorrect use of American-English), TP3 distinguish between” was considered double use of words), TP4 (“perform my work” versus “con- duct my work”), and CP6 ("I started new tasks on my own initiative” was considered double use of words). These four items were adapted to correct use of American-English, and to obtain conceptual equivalence to the original meaning. Lastly, one translator expressed issues with the answer category “seldom,” and wondered whether this should be “sel- domly” or “rarely.” However, consensus emerged that “seldom” is often used is answer scales of American-English questionnaires, and is correct use ‘of American-English, Table 1 shows the pre-final American-English IWPQ. 412. Pilottest The resulting version of the questionnaire was ministered 1040 employees of Tuts Medical Cen- ter (n=18 men and n=22 women). On average, participants were 4.5 (9.8) years of age, and worked 45,9(13.7) hours aweek, See Table 2 for an overview of the sample deseriptives Five participants (12.5%) mentioned that the instructions were not worded clearly. Two of these Participants were unsure what “how you carried out your work” meant, and three of these participants thought “conduct at work” was vague. One pati Descriptive sat geTayears mean SD) Working hoursiweck, mean SD) Primary occupation, (6) Research postion position Education position Clinical posion| ‘Admnistratve postion Highest completed education lve m (6) igh schoo degree College degree (e.g, Bachelors) Masters degree MD degree PAD degree 1. Koopmans et a. / Cross-cultural adaptation of he Individual Work Performance Questionnaire ipant felt that the use of “conduct at work” had a negative interpretation, while two other participants said that it was not negative per se, but they did not typically use those words. During the transla- tion process, conceptual issues had also arisen with these words. After discussion, the translators reached consensus thatthe phrase work” was closest to the intended meaning. Regard- ing the instructions, because few participants reported. an issue with language and an alternative was not identified, no changes were made. ‘Ten participants (25%) felt that the distinetions between the answer categories were unclear, This issue mainly concerned the distinction between “reg- ularly” and “often,” with eight participants feeling that these categories are almost the same, and could also be placed the other way around. In addition, wo participants fel that “seldom” and “sometimes” were almost the same. One participant felt that "seldom should be worded as “rarely.” Finally, two partici- pants wondered whether everyone would notice the change in answercategories forthe CWB scale. Some participants suggested to rename the answer cate- zories to “none of the time ~ some of the time — half Of the time = most of the time ~ all of the 6 to only name the extreme categories and number the middle categories, Another participant said that no matter how the answer categories are labeled, peo- ple will always have trouble distinguishing them, and they will be filled in like a visual analog scale. As no clear alternative arose during the pilot-test, and only a minority of participants reported an issue, it was chosen not to change the answer categories in order to retain equivalence to the Dutch version. Although participants stated that they had no major difficulties in understanding or answering most of how you carried out your Table? fs ofthe pilot-test sample Teal @=W) Menai) Women BSR) 352189) 338105) 439037) snBd36) 482) 19.ars) 9,500) oss) 123) 186) 0) 260) 00) 20.1) G00) 74389) san, 64180) 186) sa27, 3003) 0) 3436) 12.G00) 422) ses), 925) 31063) ears) 18673) 10356) 5@27) 123) 186) 0@1. Koopmans et a. / Cross-cultural adaptation of he Individual Work Performance Questionnaire ois the items, six items elicited the greatest number of comments, most of which were in the task perfor- ‘mance scale. Twelve participants (30%) were unsur ‘what was meant by “work result” in question TP2 kept in mind the work result I needed to achieve"), ‘They made suggestions to change “work result” to for example “work goal,” “deadlines,” oF “work ou come.” However, as there was no consensus on an altemative among the participants, nor the expert committee, the question was not changed. Thirteen participants (32.59%) thought that question TP3 ‘was able to distinguish main issues from side issues’ ‘was oddly phrased. Most of the participants said they ‘would never use the words “main issues and side issues.” and were unsure what these phrases meant, Most of the participants suggested replacing these words with “prioritize.” Therefore, the question was changed to “/ was able to set priorities.” Seventeen participants (42.5%) felt that question TP4 (“I was able to carry out my work well with minimal time and effort”) had a negative interpretation. Participants felt youcannot carry out your work well with minimal ‘time and effort. This means that you are lazy, and you take shortcuts, You need to putin time and effort to do ‘your work well. Most participants suggested chang ing the question to whether you were able carry out your work “elficiently.” Therefore, the question was changed to “I was able to carry out my work eff ciently.” Eleven participants (27.5%) indicated that question TPS (“I planned my work optimally") was strangely phrased, although they understood what ‘was meant, Suggestions for rephrasing this ques- tion were diverse, including “I scheduled my work optimally.” “I planned my work efficiently.” “I man- aged my time well.” “I could get my work done in the best way possible,” and “with planning I was able to complete all my work tasks.” The translators and authors of the IWPQ agreed that the question should be changed to “/ managed my time well.” In question CP7 (“1 to0k on challenging work tasks when these were available”), the word “these” was replaced by “they”, based on suggestion from three participants and the translators. Eleven participants (27.5%) commented on question CWB14 (“I com- plained about unimportant issues at work”). Seven ‘of these participants disagreed with the word “unim- portant,” indicating that if they complained about an issue, it by definition was not unimportant. Four of these participants wanted more specificity as to who and where they should have complained (e.g, to col- leagues orto friends, at work orat home). Therefore, ‘was decided to change the question to “complained ‘about minor work-related issues at work." In conclu sion, a (ota of five items were changed based on the pilot-test results (marked in Table 1). Almost all participants (85%) felt that all questions. ‘were applicable to their job, Two participants said that question TPS (planning work optimally) was less rel- evant to their job, because as clinicians, they had little influence on how many patients came in during the day, and which problems were presented. Two par- ticipants said CP8 (keeping job-related knowledge up-to-date) was less relevant to their job. Also, one participant said CP9 (keeping work skills up-to-date) ‘was less relevant to their job. Furthermore, five partic~ ‘pants reported reservations toanswer the CWB ques- tions honestly, because they felt the questions were slightly uncomfortable or too intense to answer. Two participants said that the CWB questions were less relevant to them, one because solving problems (neg- ative aspects) was a part of her work, and the other because she was not supposed to complain at work, Based on these few comments, it was not considered necessary to remove any questions from the ques- tionnaire, Due to the generality of the questionnaire, it was considered inevitable that some questions are less relevant to some participants than others. Al participants (100%) stated that the complete ness of the questionnaire was good. When asked, 16 participants (40%) had suggestions to expand the questionnaire to include all relevant aspects of their work performance. These suggestions mainly included determinants of individual work perfor- mance (e.g., job satisfaction, job tenure, and sleep quality), oF indicators of individual work perfor- ‘mance that were previously included, but removed during the development of the questionnaire (e.g., relationship with co-workers and supervisor(s), col- laboration with others, access to and use of supplies). Based on the suggestions, it was not considered nec~ cessary to add any new questions to the questionnaire. ‘A short questionnaire with content identical to the Dutch version was considered most important. 3.2. Measurement properties of the pre-final questionnaire Descriptive statistics of the IWPQ items can be seen in Table 1, and descriptive statistics ofthe IWPQ seales can be seen in Table 3. Almostall items showed floor or ceiling effects (>15% at the lowest or high- est answer category). At the scale level, the mean score for task performance was 2.79 (SD=0.69), 2.90 (SD=0.65) for contextual performance, and618 1. Koopmans et a. / Cross-cultural adaptation of he Individual Work Performance Questionnaire Table 3 Descriptive statistics ofthe pretnal America-English IWPQ seals Range Mean “Matin Floortfeas—-& Callngsffcis Cronbach (o-8) (SD) e000) (scored) « ‘Tsk performance 14-4 27900) 290 0 3 on ‘Comtextua performance 1504 290(068) 288 o 5 oa Counerpenutve work behavior 0-320 1.18073) 110. 5 6 039) 1.15 (SD =0.73) for counterproductive work behav- ior. The mean scale scores are comparable to scores in The Netherlands, although the mean scale score for contextual performance was slightly higher than in The Netherlands (2.90 in the USA, versus 2.31 in The Netherlands) [18]. There were no ceiling ot floor effects on the scale level. Five percent of the participants showed the highest score (4, “always") for the task performance scale, and the contextual performance scale, Five percent also showed the low- est score (0, “never”) for the counterproductive work behavior scale Tntemal consistency of the TWPQ scales. was determined using Cronbach's alpha, For the task performance, contextual performance, and counter- productive work behavior seales, the Cronhach’s alpha's were 0.79, 0,83 and 0.89, respectively (lable 3). The item-to-scale correlations were suf ficiently high (>0.40), except for item CP6 ("On ry own initiative, I started new tasks when my old tasks were completed”), which correlated r=0.24 on the contextual performance scale (see Table 1) Based on the cultural adaptation process, and the comments provided by the participants of the pilot- test, the content validity of the American-English IWPQ was judged to be good. Almost all participants in the pilot-test considered the questions to be appli- cable and relevant to their job, and all participants felt that the completeness ofthe questionnaire was 200d 4. Discussion The goal of the current study was 10 cross- culturally adapt the IWPQ from the Dutch to the American-English language and assess the question- naire's internal consistency and content validity inthe ‘American-English context. The cross-cultural adap- tation was systematically performed, resulting in an American-English version of the IWPQ that equals the original version. In general, participants were pos- itive on the comprehensibility ofthe questionnaire. A few changes were made to optimize the comprehen- sibility of the questionnaire. Here, the consideration of not changing the wording of a question in order to keep it similar to the original question, versus changing the wording of a question inorder to obtain conceptual equivalence to the original question, is important. For example, the answer category labels of the IWPQ were not changed in order to retain equivalence to the Dutch version, and because no alternative arose that was believed to improve com- prehensibility. On the contrary, the wording of task performance items 3 (‘Twas able to distinguish main issues from side issues”) and 4 (°1 was able to carry ‘out my work well with minimal time and effort”) was changed in order to improve comprehensibility In Dutch, it was chosen to give a description of “prior- itizing” and “efficiently.” as these words are hardly ever used directly. However, hased on American par- ticipants’ suggestions to improve comprehensibilty, these items were shortened to more directly ask for “prioritizing” and “working efficiently.” All par- ticipants were positive on the completeness of the questionnaire, and almost all participants indicated that all the questions were relevant and applicable to them. This indicates good content validity of the questionnaire. Thus, there appear to be no cultural differences between The Netherlands and America in measuring the concept of individual work per- formance, and the indicators used to measure the concept of individual work performance seem t0 be equivalent over these contexts. Although additional indicators of individual work performance suggested by participants in the pilot-test eg, relationship with co-workers and supervisor(s) collaboration with oth- fers, aceess to ancl se of supplies) might have heen included when developing the IWPQ from scratch in America, a short questionnaire with identical content to the Dutch TWPQ was considered most important in the current study. The generalizability of the ques- tions in the Dutch IWPQ was probably promoted by the fact that people from multiple countries (including the USA) were involved in the developmental stages of the IWPQ. forexample, during item generation [8 If the IWPQ items have Kept the same meaning after the translation, the American-English question- naire is expected to retain the same factor structure1. Koopmans et a. / Cross-cultural adaptation of he Individual Work Performance Questionnaire 67 asin The Netherlands, The sample size in the current pilot-test (n= 40) was too small to conduct a confir- ‘matory factor analysis. De Vet et al. [23] recommend a sample size of at least n= 100 to perform a reliable factor analysis, However, item-to-scale correlations ‘were examined, and were similar to the item-to-scale correlations in The Netherlands. All tems loaded suf ficiently high on the expected scales, except for item. CP ("On my own initiative, I started new tasks when my old tasks were completed”). The low loading of this item on the contextual performance scale sug- ‘gests that this item has adifferent meaning inthe USA than in The Netherlands, either due tothe translation, or due to cultural differences, However, no specific comments were made regarding this question during the pilot-testing phase, so the reason for the low load ing is unclear, Future research should administer the American-English IWPQ in a larger sample (at least 100), o that its factor structure can be examined, and the loading of the items on each scale can be examined in more detail ‘The measurement properties of the Dutch and American-English IWPQ appear to be similar, The ‘mean item and scale scores appear to be similar in both versions, although the mean scale score for contextual performance was slightly higher for the ‘American-English than Dutch IWPQ. As the IWPQ is analysed at the scale level, itis interesting to exam- ine whether more than 15% of the responses are at the lower or upper end of the scale, indicating floor or ceiling effects, There were no considerable floor of ceiling effets atthe scale level. The internal consistencies of the American-English IWPQ task performance, contextual performance, and counter- productive work behavior scales were 0.79, 0.83, and 0,89, respectively. This is similar to the Dutch ver- sion, where the scale reliabilities are 0.78, 0.85, and 0.79, respectively. The internal consistency of the American-English CWB scale is higher than in The Netherlands. 4.1, Limitations A limitation of the current study was that partie pants were aware that the questionnaire measured individual work performance, due to the informed ‘consent procedure before the study, Secondly, in the ccurrent study, a researcher was siting next to the par- ticipants while they were filling in the questionnaire. Finally, some participants reported reservations to answer the CWB questions honestly, because they felt the questions were uncomfortable or intense (© answer. All these factors may have elicited socially desirable answers, and resulted in different scores on the American-English version than the Dutch version ‘of the questionnaire. In general, we recommend leav- ing out the questionnaire title and scale names when administering the questionnaire, so that participants fare less aware they are filling in a questionnaire on individual work performance. We also recommend. that participants” answer are always anonymous and are treated confidentially. I should be guaranteed that only group level outcomes will be reported to man- agers or companies, obtained in large enough groups, so that results can never be traced back to individual participants, ‘The pilot-test in the current study was conducted in a relatively high-educated sample, with partici- pants primarily working in a pink or white collar job. This may limit generalizability of the results, to lower-educated workers, and blue collar workers. Although, in general, the translators were positive ‘on the questionnaire’s comprehensibility, applicabil- ity, and completeness for lower-educated workers, and blue collar workers, one translator had! con- ‘cerns about the use of the word “priorities” in these ‘groups. Ideally, the comprehensiblity, applicability, and completeness ofthe American-English IWPQ, as ‘well as its internal consistency and content validity, should still be examined in these groups. 4.2. Future research Although the pilot-test results indicate good internal consistency and content validity of the ‘American-English IWPQ, it is only after the cross- cultural translation and adaptation that the real cross-cultural validation takes place [23]. In a larger and more heterogeneous sample, special attention should be paid to the measurement invariance of the questionnaire in the original and the new tar- ‘get population. Measurement invariance means that ‘8 measuroment instrument, a seale, or an item, fune tions in exactly the same way in different populations [23]. This can be examined, for example, using fac~ {or analysis or item response theory (IRT) techniques. Future research should perform confirmatory factor analysis in a larger and more heterogeneous sam- ple, and examine if (and if so, why) item CP6 loads insufficiently high on its original dimension. IRT techniques are also a powerful method with which to detect differential item functioning (DIF), by comparing the item characteristic curves of the items in the original version and the translated ver-os 1. Koopmans et a. / Cross-cultural adaptation of he Individual Work Performance Questionnaire sion [23]. This can give insight into whether the difficulty of an item has changed in the original and translated version. Future research should also further examine the reliability and construct valid- ity of the American-English IWPQ. In order to use the IWPQ for examining the etfectiveness of interventions, procedures and strategies to maintain, improve, or optimize individual work performance ata group-level, future research should also address responsiveness to change and minimally important change, as well as predictive validity and criterion validity of the American-English TWPQ. 5. Conclusion The cross-cultural translation and adaptation of the IWPQ from the Dutch to the American-Engli language was conducted without major difficulties. ‘The comprehensiility, applicability, and complete- ness of the translated version of the IWPQ were appraised positively. Also, the study showed posi- tive results conceming its intemal consistency and content validity: Fture research should further e and responsiveness to change of the questionnaire in a larger and more heterogencous sample. After fur- s-cultural validation, the American-Engl TWPQ may be used to measure, for example, the effectiveness of workplace interventions on indi- vvidual work performance in an American-Engli speaking context Acknowledgments We wish to thank Kimi Uegaki, Tammy Rubin- stein, Fenna Leijten and Nico Pronk for their help in translating the questionnaire, the EMGO+ Insti- tute for Health and Care Research for providing the travel grant, Tufts Medical Center for their hospital ity, and the Tufis Medical Center employees for their participation in the pilot-test References [1] Baropcan Commission, EUROPE 2020: A strategy for st, stainable and inclusive growth, Brussels: Euro ean Commission; 2010 Available fom: hipee europa. euleurope2020" [2] Campo Modeling the performance rection problem jn indonrial snd engunizatonal psychology. In: Dunete BI oy 151 6 m 8 p 0, mm 14 13) na) nist 118) 7 is, 9, MD, Hough LM, editors, Handbook of Indust and Orpnizational Psschologs. Palo Alta, CA: Consulting PSY shoots Pres; 1990, pp. 687-732, ‘Austin JT, Villanova P. The trio problems: 1917-1992, Soural of Applied Psychology 1992:71(6)836-74 ‘Viswestaran C, Ones DS, Perspectives on models of job perfrmnee Intratonal Jura of Selec and Asses ‘ent 2000 8(4:216:26, Rotundo M, Sackett PR. The relative importance of ask. citizenship, and counteprsuctive performance to global ‘atngs of performance: A polcy-captring upproach. Jour ‘aa of Applid Psychology 200258716680. ‘Koopmans L, Bernards CB, Hildebrandt VH, Schaufeli WB, De Vet HCW, Van der Boek Al- Concepual frame works of indvidusl work performance: Asssemtic review, Journal of Ovevpations) and. Environmental Medicine 2011S318):856-66 Borman WC. Motwridlo SJ. Expanding the criterion domain to include elements of contextual performance. Im: Schmitt N, Borman WC, editors. Personnel Soection Jn Organizations. San Francisco, CA: Josey Bas; 1993, 9.7198 Koopmans L, Bernards CM, Hildebrandt VH, De Vet IEW, van der Beek A. Measoring indivi work perfor ‘mance: Identifying ad selecting indiewors. Work: Joural ‘of Prevention, Assessment & Rehabilitation 201315(). Willams LJ, Anderson SE. Job satisicton snd org: sizatonal commitment as predictors of erganizational ‘itzenship and in-rle behaviors, Joumal of Management 10011713}:601-17 PodsakofY PM, MacKensie SB. A second generation measure of organizational citizenship bebavior Indians Univesity, Bloomington, 1989 ‘an Scat JR, Moti SI. Inerpersonlfciliation and {dedication as separate acess of contextual performance Soural of Applied Psychology 1996:81(5)52531 Benne RJ, Robinson SL. Development of measure ‘of workplace deviance, Joumal of Applied Psychology 200038513):349-60. Spector PE, Fox 8, Penney LM, Braursema K. Got A Kessler S. The dimensionality of counteprouctivity: Are all countermdactive havior crested equal? Touma of ‘ocatonal behavior 20066844660, Reilly MC, Zhrozek AS, Dkes EM. The validity snd repro Aeiility OF «work praditvity and activity impairment instrament, Phiemacooconomics 199545). Leror , Amick BC, Rogers WH, Malspes 8, Bungay K, Cyn D. The work Hinitations questionnaire, Medial Cre 200133941): 72.85, Kessler RC, Burber C, Beck A. Berglund P, Cleary PD. ‘MeKenasD, etl. The world health organization health and ‘work performance questionaire (HPQ). Journal of Occ pation snd Enviromental Medicine 200885:156-74 Dalal RS. A meta-analysis ofthe relationship between orgs ‘zation citizenship behavior nd counerpodutive work. behavior Journal of Applied Prehology 2008901241 Koopmans L, Bernards CM, Hildebrandt VH, van Buren S.vunder Beck Al de Vet HCW. Development ofan inivid- ‘ual work performance questionnaire ntrationalJoural ‘of Productivity and Performance Management 2013241) 628, ‘Koopmans L, Bemsards CM, Hildebrandt V, Van Buuren S, Van der Beck AJ, De Vet HCW. Improving the individ tual work performance questionnaire using rasch analysis. Soural of Applied Measurement 2014:18(2):160-75,Po} pu 1. Koopmans et a. / Cross-cultural adaptation of he Individual Work Performance Questionnaire 619 Koopmans L, Bernas CM, Hildebrandt VH, De Vet HW, Vin dor Beek AJ Construct validity ofthe individ ‘work performance questionnaire. Joural of Occupational and Environmental Medicine $63):331-7. Beaton D, Bombardier C, Escorpizo R, Zhang W, Lacaille D, Boonen A. et al. Measuring worker productivity Frameworks and measures, The Journal of eheumatology 2009536(9)2100.5, ultingse M, Alonso J, Apolone G. Leplege A Sul- livan M, Wood-Dauphince S. etal Translating heal 3 pa status quesionnsies and evaluating their quality: The IQOLA project appre Journal of Clinical Epidemiology 199g;51011)913-23, De Yet HCW, Terwee CB, Mokkink LB, Knol DL. Mew: surement in Medicine: Cambridge University Press, 201 CCronhuch 1. Coefficient alpha and she intemal stacture of ech Pyehometrka 1981 16297-333,