MSP Research Note

CSQ Reliability, Validity and Norms

Introduction
This research note describes the reliability and validity of the CSQ. Evidence for the reliability and validity of is presented against some of the key the criteria in the EFPA Review Model for the Description and Evaluation of Psychological Tests (Bartram, 2002).The EFPA Review Model was produced to support and encourage the process of harmonising the reviewing of tests. It provides a standard set of criteria to assess the quality of tests. These cover the common areas of test review such as norms, reliability and validity.

Reliability
Internal consistency reliabilities Table 1 shows the CSQ scale reliabilities. The internal consistency for the questionnaire as a whole is 0.92 and the median internal consistency for the CSQ scales is 0.78 which is which is defined as good by the EFPA review model (Bartram, 2002). The CSQ scale sten score SEms all in the range of 0.8-1.1. This means that there is a 68% likelihood that the person’s true score on the scales will be about 1 sten either side of the observed score. The sten score SEm band around the CSQ total score is smaller (about .5) because the reliability is higher.

then we can be reasonably certain that they are independent. Risk-Taking 0.07 0. A correlation needs to be divided by the square root of the product of the two variables’ reliability to determine what the correlation between the two variables would be if the variables’ reliabilities were perfect.54 Construct Validity Scale intercorrelations Table 2 shows the intercorrelations of the CSQ scales. There are moderate to high correlations between the scales ranging from 0.56 SD 5.78 0. it is necessary to correct the correlations for unreliability.82 2. it is apparent that there is a high degree of overlap between the Original.81 2.49 0.47 In order to determine how well a questionnaire differentiates between the different dimensions it is designed to measure. Intercorrelations of CSQ scales (n = 4.49 Achieving 0.41 37.79 0.162) Scale Original Risk-Taking Adaptable Assertive Achieving Total Score Alpha 0.15 22.56 Assertive 0.98 6.52 37.88 6.65 Adaptable 0. Internal consistency reliabilities for CSQ (n = 4.Table 1.51 0.92 Mean 36. Using 50% as a benchmark.26 0.70 0.95 1.78 0. If two scales share less than 50% reliable variance.45 31.84 0.75 0.27 5.26 to 0.162) Scale Original Risk-Taking Adaptable Assertive All correlations are significant at 0. page | 2 . Table 2.01 level (2-tailed). The remaining scale pairs shows a reasonable degree of independence (Table 3).98 Raw score SEm 2.50 Sten score SEm 0.70 178.70 0.78 0.55 0.65 6. Risk-Taking and Adaptable scale pairs.23 5.48 35.57 0.86 2.98 0.70.89 2.

31 Standard error of difference The Standard Error of Difference (SEd) helps us determine the size of the gap that you need to see between a person’s scores on any two scales before you can conclude that the difference is real.51 55.04 4.118 respondents (see Table 5).06 4. Standard error of difference of CSQ scales (n = 4. Percentage of common reliable variance for CSQ scales (n = 4.07 Assertive 3. Table 4. This is equivalent to a difference of 3 Stens.Table 3.75 11.02 Adaptable 4. One factor was extracted accounting for 67% of the variance indicating that creativity and problem solving style measured by the CSQ is a single construct.46 36. Efficiency and Conformity.01 4. The KAI has 3 subscales: Originality. Communality values were moderate to fairly high.82.61 56. The SEd depends on the reliability of the scales – the higher the reliability the smaller the SEd is. 1994).03 4.03 Achieving 4. The variables were on the whole well-defined by the factor solution. The KAI places people on a single global dimension where one end of the dimension represents ‘innovators’ and the other end represents ‘adaptors’.162) Scale Original Risk-Taking Adaptable Assertive Risk-Taking 4. All the SEds in Table 4 are around 4 so a difference of around 8 or more in CSQ raw scores constitutes a real difference.11 39.71 Assertive 59. The Kaiser-MeyerOlkin measure of sampling adequacy was 0. If there are two full SEds between the scores on two scales.97 Achieving 42. then there is a 95% likelihood that there is a real difference.40 43.98 4.08 4. The CSQ and the KAI were administered page | 3 .31 Adaptable 88. Relationship to other measures The Kirton Adaptation-Innovation Inventory (KAI) is a 32 item questionnaire which provides a measure of creative style (Kirton.6 required for a good factor analysis.162) Scale Original Risk-Taking Adaptable Assertive Risk-Taking 71. well above 0.02 Factor analysis Principal factors extraction with varimax rotation was performed on the CSQ scales on a sample of 2.

02 0.67) and the CSQ RiskTaking scale correlates significantly with the KAI Conformity Scale (0.88 0. The sample spanned 28 nationalities.17 Conformity 0.61* 0.03 0.2 0.41* 0. This is based on a sample of 1.78 0. page | 4 .63* 0. The pattern of correlations confirms that CSQ measures important elements of creative style.67* 0.35* 0.78 0.56* KAI Total 0.28* 0.118) Scale Original Risk-Taking Adaptable Assertive Achieving F1 0. Combined ratings shown in the table are the sum of the manager’s assessment and the test taker’s self-assessment. Table 6 shows correlations between the KAI total score and subscales and the CSQ total score and subscales. Regression analysis indicated that the CSQ subscales account for about 65% of the variance in the KAI total score.48* 0.37* 0. poor) and to assess their own performance.11 0. good.58* 0.49* 0.58). satisfactory.07 0.61 0.33* -0.84 0.960 respondents who completed the CSQ on the internet.67 Communality 0.05 -0.59* 0.80 0. Correlations between CSQ and the KAI (n = 126) Scale Original Risk-Taking Adaptable Assertive Achieving Total Score * p <.45 The CSQ total score and the KAI total score correlate significantly at 0. Respondents were asked to report how their manager assessed their performance at their last performance appraisal using a 4-point scale (excellent.to a sample of 126 managers and professionals studying for a MBA at Manchester Business School.70 0. Factor loadings and communalities for principal components extraction on CSQ scales (n = 2.69* Criterion-related validity Table 7 shows the correlations between CSQ scale scores and job appraisal ratings.69 which is defined as good by the EFPA review model.54* 0.63 0. Table 5. The CSQ Original scale correlates significantly with the KAI Originality scale (0. Table 6.01 Originality 0.2 0.75* Efficiency 0.

shared variability = 0.64 0. Unique variability = 0.30 0.07. For example.61 1.68 0.12 0.01** 0.07 in shared variability. p < 0.02 -0.04 -0.80 R2 Adjusted R 2 0. Altogether.34 5.1-0. the upper limits for the validity of personality variables against overall work performance variables are in the range of 0. 14% of the variability in job performance ratings was predicted by knowing scores on the five CSQ scores.18 0.06** Mean Standard deviation 6.01.44 1.58 0. a page | 5 Unique Beta B . A standard multiple regression was performed between managerially and self-assessed job performance combined as the dependent variable and the CSQ scales as the independent variables.24 0.30 0. Table 7 displays the correlations between the variables.4.07.14a 0. R2 and adjusted R2 .001.16 -0.25 to 0.14 0.96 5.00 0.58 0. Two CSQ scales (Original and Achieving) contributed significantly to the prediction of job performance ratings (sr2 = 0.24 0. the unstandardised regression coefficients (B). the semi-partial correlations (sr2) and R.97 5.00 0. Table 7.06 1.07). and the five scales in combination contributed another 0.63 6. Regression of CSQ scale sten scores on job performance ratings (n = 1.4 between job performance and all four factors.57 0.11 -0. These correlations are consistent with those reported in the literature for personality assessment instruments. 1954) = 68. For the combined ratings.89.02 0.48 Intercept 4. Robertson (1997) notes that even with meta-analytic corrections.09 0.51 0.There are statistically significant correlations at 0.38 R ** P<0.00 0.06 0.26 0.36 0.70 0.50 0.02 0.72 2.08 5. F(5.81 1. Regression analysis was used to help understand the contribution of the different areas of emotional intelligence to job appraisal ratings.69 1.960) Combined Assessment (DV) Risk-Taking Adaptable Achieving Assertive Original Original Risk-Taking Adaptable Assertive Achieving 0.51 0. R for regression was significantly different from zero.

The mean age of the sample was 33 with a Standard Deviation of 11. The norms are presented in Table 8. Figure 1.000 respondents who completed the questionnaire over the Internet between July 2002 and January 2004. A sample was created with equal numbers of men and women between the ages of 16 and 64. Age distribution of CSQ international norms sample (n = 4.000) page | 6 . The age distribution is shown in Figure 1.Norms CSQ norms are based on a large international sample of 4.

23 5.56 5.41 37.48 35.15 22.45 31.000) Stens Scale 1 Original Risk-Taking Adaptable Assertive Achieving Total Score 10-24 10-18 10-25 10-21 10-23 50-131 2 25-27 19-21 26-28 22-25 24-27 132-143 3 28-29 22-24 29-31 26-29 28-30 144-153 4 30-33 25-27 32-34 30-32 31-34 154-166 5 34-36 28-30 35-37 33-35 35-37 167-178 6 37-38 31-34 38-39 36-38 38-40 179-190 7 39-41 35-37 40-42 39-40 41-43 191-201 8 42-44 38-40 43-44 41-43 44-45 202-212 9 45-46 41-43 45-46 44-45 46-47 213-221 10 47-50 44-50 47-50 46-50 48-50 222-250 Original Risk-Taking Adaptable Assertive Achieving Total Score 36.98 Scale Mean SD 7|Page .27 5.70 178.65 6.98 6. CSQ general population norms (N = 4.Table 8.52 37.

be: European Federation of Psychologists’ Associations. D. EFPA Review Model for the Description and Evaluation of Psychological Tests: Notes for Reviewers. UK: Thomson Learning. London. Robertson. myskillsprofile www. © 2004-2010. T. www.efpa. (2002). (1994). Adaptors and Innovators: Styles of Creativity and Problem Solving. I. Kirton.com page | 8 .References Bartram.. M. J. Personality and Work Behaviour (1997) Keynote Address to 2nd Australian Industrial and Organizational Psychology Conference.myskillsprofile.

page | 9 .