A Bayesian Approach to Measurement Bias in Networking Studies

Ling Zhu Scott E. Robinson April 11, 2012 Ren Torenvlied e

Paper prepared for the 70th Annual National Conference of the Midwest Political Science Association, April 14, 2011, Chicago, IL. Department of Political Science, University of Houston, 447 Philip G. Homan Hall, Houston, TX, 77204-3011; lzhu4@central.uh.edu. Bush School of Government, Texas A&M University, 2010 Allen Building, 4220 TAMU College Station, TX 77843-4220; srobinson@bushschool.tamu.edu. Institute of Public Administration, Leiden University, Pieter de la Court Building Wassenaarseweg 52 2333 AK Leiden, Netherlands; r.torenvlied@fsw.leidenuniv.nl.

Abstract The study of managerial networking has been growing in the eld of public administration. Coincident with this interest in managerial networking is an interest in the use of selfreported survey data to measure managerial behavior in building and maintaining networks. The predominant approach is to use self-reported ordinal scales, aggregated through factor analytic techniques, to generate a measure of networking activity. However, when public managers use ordinal responses to describe their networking activities, they may answer these survey questions in ways subject to a variety of response biases. Hence, ordinal scales may lead to biased measurements that misrepresent managerial networking. We propose a Bayesian item-response theory (IRT) measurement strategy to infer how managers in opensystem organizations interact with dierent external actors and organizations. To quantify the latent ranking-order of the eort of managerial networking, we use a Bayesian approach to select one-dimensional scales from a bank of multiple survey items. Using 10 ordinal survey items in a mail survey of 1026 American hospitals, we demonstrate the advantage of using the Bayesian IRT model over factor analytic models.

Introduction

The study of networking activities is growing in the led of public administration(Agrano and McGuire, 2001; Isett et al., 2011; OToole and Meier, 2011). The dominant approach to measure managerial networking activities is to apply factor analysis techniques to selfreported ordinal survey data (Agrano and McGuire, 2003; Meier and OToole, 2005; Thomson, Perry and Miller, 2009). Factor-analytic methods, combined with survey research methods, impose two major challenges to measurement validity: (1) the compatibility issue (i.e. survey respondents may understand the same question in dierent ways) (Brady, 1985; King and Wand, 2007; King et al., 2004) and (2) social desirability bias (i.e. the tendency for individuals to respond in socially desirable ways producing a common source bias) introduced by perceptual survey items (Donaldson and Grant-Vallone, 2002; Doty and Glick, 1998; Graham and Collins, 1991; Podsako and Organ, 1986; Schwarz, 1999). As Meier and OToole (2010) note, the eld of public administration has rushed into performance appraisal and performance management without applying measurement theory and methodology that deals with measurement bias. The absence of both theoretical and methodological rigor of measurement, according to Meier and OToole (2010), is the Achilles Heel of public administration research. Indeed, the scholarly search for general theories heavily depends on empirical studies that validate analytical propositions. Measurement quality aects our ability of testing causal relationships, making statistical inferences, and more generally, our understanding of administrative behavior. Improving measurement that maps administrative and managerial behavior, thus, is critical to theory-building. We address this enduring, yet overlooked, problem of measurement bias in the study of networking activities. Drawing from the Item Response Theory (IRT) (Embretson and Reise, 2000; van der Linden and Hambelton, 1996), including non-parametric Mokken Scale analysis (Mokken, 1971; van Schuur, 2003, 2011), we propose a Bayesian approach to infer the latent dimension of managerial networking. The Bayesian approach to measurement conceptualizes the true eort level of managerial networking as a latent variable, which is not directly observable. Theoretically, managerial networking activities are driven by timebudget constraints (i.e. diculty in concentrating time-allocation on one networking node) and time-allocation decisions are made based on returns to investment in dierent networking dimensions (i.e. the discrimination of one networking node from all the other nodes). A Bayesian model, furthermore, is proposed to deal with uncertainty in parameters caused by response bias. We argue that the Bayesian approach of measurement is more attractive than the factor analytic methods because it relaxes the assumption that respondents truthfully report their networking activities though care must be taken to consider dierential item functioning (dif) (Embretson and Reise, 2000). The Bayesian IRT approach, moreover, extend the non-Bayesian IRT models in that it is exible to handle cross-item heterogeneity and models uncertainty in parameters. The Bayesian IRT approach, additionally, improves measurement reliability via simulation-based inferences (Fox, 2005, 2010; Jackman, 2009). Using 10 survey items on networking activities in professional health care organizations 3

(American hospitals), we illustrate the statistical procedure of computing the Bayesian IRIT measure. We also demonstrate that the Bayesian approach to measurement helps to reduce measurement bias associated with the classic factor-analytic methods and is particularly useful when the number of observations is relatively small.

Challenges of Measuring Networking Activities

Students of public administration are often interested in studying causes and eects of managerial networking (Meier and OToole, 2005; Robinson, 2006). Research into networks and policy implementation, specically, build upon assessing latent traits such as the level of activity in creating and maintaining networks, the eort in network interactions, etc. One common problem is that such latent traits that network scholars intend to measure do not have well-established units of measurement. As large-scale research into networking management mushrooms, the issue of lacking clear-cut measurement becomes salient. Currently, large-scale studies on networking management are sparse (Meier and OToole, 2005; Robinson, 2006). A few prominent examples are marked by the use of survey methods to capture self-reported collaborative relationships or managerial activities. For example, Meier and OToole (2001) survey top-level administrators in Texas school districts and ask superintendents to self-report their frequency of contact with dierent nodes. The article extracts a single continuous factors from the reported contact frequencies as the basis for their long series of studies of the impact of networking among Texas school districts including applications to such areas as disaster response OToole et al. (2006). McGuire (2001) and Agrano and McGuire (2003) survey city managers about their vertical and horizontal collaborative activities. The measurement strategy within these studies focuses on the nature of collaborative activities rather than frequency. Respondents reported collaboration for information sharing, project-based partnerships, or other specic activities. The shift in focus emphasizes the content of a collaborative relationships and the nature of the work conducted in a collaborative manner rather than the frequency of contact. Finally, one can focus on subjective attitudes related to partnership from the perspective of a collaborating organization Thomson, Perry and Miller (2009). Thomson et al. measure collaboration based on a series of attitudinal questions related to collaborative experiences. These attitudes are groups into ve dimensions based on seventeen survey questions related to experiences with undierentiated partner organizations. These ve dimensions contribute to a single dimensional indicator of collaboration. Like with the Meier and OToole measurement approach, a series of ordinal component measures (this time of attitudes related to collaboration experiences) are combined through factor analysis to create a single measure of collaboration. Work has also emerged on European administrative networks. Andrew et al. (2011) survey English local government managers and measure external networking based on selfreported interactions with external stakeholders to their organizations. Torenvlied and 4

Akkerman (2012) measure Dutch primary school principals managerial networking based on self-reported survey items measuring how frequent they interact with external networking partners. Although the empirical contexts of these large-scale studies are heterogeneous (i.e. different organizations, dierent policy issue areas and dierent countries), the similar research design leads to a common set of measurement problems. Multiple survey items, among other things, are often scaled based on factor analytic methods and combined into a networking index with the assumption that the numerical weights assigned to each survey item are comparable across all respondents. The potential heterogeneity across respondents is a typical source of variation in response data that needs to be accounted for in a statistical response model (Fox, 2010, 3). Survey items on managerial networking are extremely vulnerable to this validity threat in that both managers personal traits and their organizational contexts might induce a specic networking activity and aect its frequency. A factory index without accounting for the cross respondents heterogeneity will be a biased measure, which is not compatible across all respondents. Parallel to the compatibility issue is the challenge of scaling dierent behavioral items into one dimension that maps the latent traits of managerial networking. The theoretical literature on managerial networking does not oer much guidance on the dimensionality of networking activities. Empirical research, moreover, produced mixed ndings. Earlier studies often treat networking as a single-factor measure, with the statistical assumption that dierent survey items share the same underlying distribution (Geletkanycz, Boyd and Finkelstein, 2001; Hicklin, Meier and OToole, 2008; Meier and OToole, 2001). A few recent studies, nevertheless, discover the multiple dimensions of managerial networking (Agrano and McGuire, 1999; Akkerman and Torenvlied, 2011; Torenvlied et al., 2012). Whether managerial networking has a single trait dimension or contains multi-dimensional traits remains to be an unsolved puzzle. Both the factor-analytic technique and the non-parametric cumulative scaling technique (Mokken Scale), in addition, take the explorative strategy by letting the survey data speak, and assume that each behavior item is correctly measured. In other words, measurement scales are produced with the assumption that managers truthfully report their activities of managerial networking. Recent studies show that self-reported survey items are vulnerable to various reporting bias. Meier and OToole (2010) demonstrate that managers might overreport or underreport particular networking activities due to the social desirability of the survey item. As such, common resource bias is likely to occur (Donaldson and GrantVallone, 2002). Henry, Lubell and McCoy (2012) nd that self-reported survey items are likely to have recall bias, namely, respondents are unlikely to remember their entire list of network nodes and report accurately how much they networked with each node. These most recent studies, nevertheless, are long on description and short on prescription of how to quantify managerial networking, accounting for measurement bias embedded in the survey data.

A Bayesian Item Response Approach to Measurement Bias

We propose a Bayesian Item Response approach to model ordinal survey items of managerial networking. Our approach builds on the classic Item Response Theory (IRT) and the cumulative scaling technique, but adds explicit conceptualization of errors in the measurement. Bayesian inference is then used to improve measurement reliability.

3.1

Ordinal Response Data and The IRT Approach

The IRT approach of scaling (e.g. Mokken scale) conceptualizes that the probability of a particular response to a survey item is a function of characteristics of the respondent and the survey item (Hemker, Sijtsma and Molenaar, 1995). In the context of analyzing dichotomous item responses, the answer a person gives to a question is interpreted as a dominance relationship between the person and the question (the item). The person dominates the item if she gives the positive response, and the item dominates the person if she gives the negative response (van Schuur, 2011, 16). With a large number of survey items and respondents, one can analyze four types of dyadic dominance relationships: item-to-subject, subject-to-item, item-to-item, and subject-to-subject. Without measurement errors, this scaling approach can produce a transitive rank-order among all items based on the diculty parameter and a transitive rank order among all items based on the discrimination parameter. The latent dimension of ability (or other latent traits, such as eort, ideology, attitude, etc.), therefore, is inferred and scaled based on the cumulative probability of giving a positive response idenned by the diculty and discrimination parameter. When applying the IRT approach to ordinal responses, the model essentially becomes a cumulative rating scale model or a partial credit model (PCM) (Anderson, 1997; Masters, 1988; Samijima, 1969). Ordinal response items in networking studies often generate a scale based on ordered categories. The survey question may ask a respondent using letter grades, A, B, C, and D ( ordered from low to high), to indicate the eort of his networking activities. A survey item may also ask a respondent the frequency of her networking activities, rendering a scale, ordered as never, yearly, monthly, weekly, daily, and more than once per day. Attitude items, in addition, normally produce an ordered scale ranked by strongly disagree, disagree, agree, and strongly agree. Compared to the dichotomous survey items, ordinal items, or more generally, polytomous items increase the statistical information (Ostini and Nering, 2006) that researchers can use to infer managerial ability or eort of networking. Let the latent dimension of networking eort for i managers be denoted by i , one can use j items to denote dierent networking activities. The j survey items then will produce a stack of item-subject matrixes, illustrated as Figure 1. As Figure 1 shows, for each survey item, respondents report their choices among the 6 categories, which will produce a density distribution based on all the reported choices for that particular item. For Item 1, the most 6

likely choice is 6, and for Item j, the most likely choice is 3. [Figure 1 About Here] One can then use the stack of survey items to estimate the latent eort of managerial networking. Specically, for a survey item j (networking node), the probability of a manager i to report a specic response k in the ordinal k-choice category k = 1, 2, 3.. is dened by the dierence between the probability of responding in (or above) choice k and the probability of responding in (or above) the next choice, k + 1 (Fox, 2010, 13). P (Yi,j = k|i , aj , bj ) = P (Yi,j k 1|i , aj , bj ) P (Yi,j k|i , aj , bj ) (1)

The cumulative probability of choosing the lowest category and above is assumed to be 1 and the probability of choosing above the highest category is 0. As for a k-choice ordered scale, there are (k 1) cut points to dene the ordinal scale. Hence, equation (1) species the two probability values with respect to cut point k 1, and k. In equation (1), aj and bj are the discrimination and diculty parameter, respectively. i is the latent dimension at interest. The two probabilities in equation (1), further, can mathematically be represented by the logistic cumulative distribution function (Bradlow and Zaslavsky, 1999; Fox, 2010; Fumiko, 2008).

P (Yi,j = k|i , aj , bj ) = P (Yi,j k 1|i , aj , bj ) P (Yi,j k|i , aj , bj ) exp(aj (i bj,k )) exp(aj (i bj,k1 )) = 1 + exp(aj (i bj,k1 )) 1 + exp(aj (i bj,k ))

(2)

3.2

Considering Measurement Bias: A Bayesian Approach

Reporting bias and the issue of cross item compatibility can both change the perfect rankorder estimated based on the diculty and discrimination parameter. The item response theory conceptualizes measurement bias as the violation of a transitive relationship between items and subjects. Bias, counted as reporting errors (or wrong answers ), thus can be estimated through evaluating the homogeneity of individual items based on pairwise information extracted from each item pairs (van Schuur, 2011). Figure 2 illustrates how the pairwise association may look like between two ordinal survey items. In Figure 2, both items are coded based on a 1-to-6 ordinal scale, and each circle indicates the frequency of all possible pairwise values, whereby a large circle refers to high frequency and small circle refers to low frequency. Ideally, if Item I and 2 are created to measure the same latent construct, the association between the two items should only be caused by their common associations with the latent factor. The pattern shown in Figure 2 obviously demonstrates a lot of noisy information. Among all 36 possible pairwise values between the two items, 4 pairs (16, 25, 32 and 62)are not observed. The concentration of the 16 pairwise values, when 7

both Item 1 and Item 2 take values greater than 2, clearly demonstrate the deterministic relationship between the two items. [Figure 2 About Here] The challenge to map the latent trait based on data variations generated by survey items is that we do not know the true measurement score of the latent construct. When observing noisy data as shown in Figure 2, it is dicult to handle the uncertainty associated with measurement errors. The Bayesian IRT approach becomes particularly appealing to handle such measurement issue, because neither the latent traits, nor the item parameters need to be treated as deterministic. Building on the likelihood function expressed by equation (2), we add a random errorterm j to represent measurement bias. As such, the probability of manager i reporting a particular choice kj for survey item j can be expressed as a cumulative logistic density below the the latent cut point k,j associated with choice kj . P [Yij = kj ] = P [aj i bj + j kj ] = P [aj i bj (kj j )] = 1 Z(k,j F (i , aj , bj ))

(3)

To simplify the mathematic notation, in equation (3), we use Z() to denote the cumulative logistic density function, and F (i , aj , bj ) to denote the linear link function for Yij dened by latent trait i , the discrimination parameter aj , the diculty parameter bj . k,j is the estimated cut point specic to the grade value k, and item j. Reporting bias j , in additional, also determines the actual estimated value for k,j . In the context of estimating a full Bayesian model, all the parameters in F () are estimated quantities, thus carry measurement uncertainty in parameters. Specically, the latent dimension of managerial networking and the the characteristics of survey items are all unknown and we are interested in learning (inferring) their densities jointly based on the assumed likelihood function, the prior information on their distribution, and the observed response data. Using (, ) to denote the joint posterior density that we want to infer, whereby refers to the latent trait of managerial networking, and refers to the vector of diculty parameters, and refers to the vector of discrimination parameters. The marginal posterior density is learned based on using observed data to update prior assumptions about how these unknown parameters are distributed. Formally, the posterior density is given as: (, , |Y ) L(Y |, , )p()p()p() (4)

Using the Bayesian approach, one not only can recover the point estimation (e.g. posterior means ) of managerial networking (i ), but also can obtain the condence intervals of the measurement score (for example, see Treier and Jackman 2008). In the next section, we fully 8

specify each component of the proposed Bayesian IRT model using empirical survey items on professional health care organization.

4
4.1

An Empirical Application: Managerial Networking in American Hospitals Data

The empirical data we use to illustrate the Bayesian IRT approach to measurement are collected from a mail survey of 6,000+ professional health care organizations, including general hospitals, specialized hospitals, mental health clinics, childrens hospitals, university-owned medical centers, rehabilitation centers, and acute long-term care organizations. The survey was on eld in the spring of 2011 and generated 1026 responses, rendering an overall response rate of 16.14%. Although the overall response rate is not high, it is not surprisingly low compared to other large-scale surveys on American hospitals.1 We perform a logistic regression analysis to check if response rates vary substantially by organizational type, region, state, service type, and organizational size (see Table 1). Our sample does not vary substantially along all these indicators excepting organizational type.2 Table 2 reports survey response rates by dierent organizational types. In our sample, there are 343 government-owned hospitals(primarily non-federal hospitals), 483 non-prots, and 195 for-prots. [Table 1 About Here] [Table 2 About Here] The survey uses a name-roster method, providing top-level hospital administrators a list of 25 networking nodes, whereby 6 are internal to the organization and 19 are external to the organization. Table 3 reports summary statistics of the full list of 25 networking nodes. We use the 11 nodes (see Figure3) for external networking patterers to estimate the Bayesian measure for managerial networking. Each survey item is measured by a 1-to-6 ordered scale ranking the frequency of interactions between a manager and her networking partners. The order scale is dened as: 1=never, 2=yearly, 3=monthly, 4=weekly, 5=more
For example, the Agency for Healthcare Research and Quality (AHRQ under the Department of Health and Human Services) released the 2012 Hospital Survey on Patient Safety Culture among American Hospitals, and data were reported from 1128 hospitals, which covers a comparable proportion of hospitals to our sample. As for all surveyed hospital sta respondents, the participation rates based on two major areas are relatively low (surgery-10% and Medicine: 12%) (see, at http://www.ahrq.gov/qual/hospsurvey12/). 2 Although service type and organizational size are signicantly correlate with the survey response rate, the coecients are not very large. Organizational type, however, is a strong predictor of the response rate. Because we only use the survey items to demonstrate the computation of our proposed Bayesian measure and then compare the Bayesian measure to the conventional factor analysis, the two measures are baring the same sample errors. If the survey data were used to analyzing the relationship between managerial variables and organizational performance, one needs to consider weighting survey responses or controlling for dierent organizational types.
1

than once a week, and 6=daily. We did not use all 19 nodes of external networking primarily due to the missing data in a few nodes, such as Volunteer Board, Hospital Foundation, Union Representatives, etc. All the selected 11 nodes are important political stake-holders to hospitals. Substantively, we focus on the 11 external nodes to construct a measure of external organizational networking. This approach is analogous to that used by Meier and OToole, Agrano and McGuire, and others in dening managerial networking as an activity that crosses organizational boundaries. [Table 3 About Here] [Figure 3 About Here]

4.2

Prior Specication and Estimation

We use the R and JAGS program to implement the Bayesian ordinal IRT model. We specify uninformative priors for aj , and bj,k . Because the latent dimension of managerial networking, i and the discrimination parameterj , and the diculty parameter betaj,k are simultaneously estimated, we set j to be truncated at 0 and constrain the mean of thetai to be 0 for the purpose of identication (Bafumi et al., 2005; Curtis, 2010; Fox, 2010). Prior distributions are specied as following: N (0, 1.4) N (0, 6.25) N (, 2 )

(5)

We do not set i to be a standard normal distribution, because we want to infer variance introduced by reporting errors. Hence, we specify the hyper priors for i to have a mean of 0 and precision of 0.01. The precision parameter is dened as a gamma distribution, with shape=0.5 and rate=0.5. In other words, we xed the mean value of managerial networking to be 0, and use the estimated variance to capture uncertainty in the latent dimension. The Bayesian IRT model is estimated by running 100,000 iterations of Markov Chain Monte Carlo (MCMC) simulation.3 To initialize the model, we use JAGS to randomly generate initial values to all the parameters. Model results are reported based on burning the rst 10,000 iterations, and restoring every 10th sampled values for j , j,k , , and 2 . Post estimation diagnostic analysis did not nd signicant evidence for non-convergence (see the sample trace-plots and density plots in Figure 4).4
We rstly run the MCMC simulation for 50,000 iterations. Model convergence, however, is not ideal, due to high serial autocorrelations in parameters related to some networking nodes. Hence, we run an additional 50,000 iterations and the model convergence has been substantially improved. 4 In total, our model produce ten parameters indexed by each networking node (j), and (j 1) k (9 6 = 54) parameters because we constrain j,1 to be 0. Figure 4 illustrates the density plots of the discrimination parameters associated with rst 3 networking nodes (State Medicaid, State Medicare, and
3

10

[Figure 4 About Here]

4.3

Comparing Factor Analysis and the Bayesian IRT Approach

The proposed Bayesian IRT model (Generalized Partial Credit Model) returns a full set of discrimination parameters, diculty parameters, and recovers the marginal posterior distribution of the latent dimension, i . Table 4 summarizes the posterior means of the 10 discrimination parameters and latent dimension of managerial networking. To compare the Bayesian IRT estimation with the factor analytic model and the non-Bayesian IRT model, we perform factor analysis and non-Bayesian item response analysis and report the results in Table 5.5 Parameter Reliability. Overall, our IRT model produces a reliable estimation of item discrimination parameters. Comparing the posterior means and medians of the 10 networking nodes in Table 4, all 8 networking nodes, excepting Insurance Companies and National Hospital Association have very large values of the discrimination parameter. Although the mean coecients for Insurance Companies and Local Hospital Association are relatively smaller than all the other 8 networking nodes, cross-item heterogeneity is reasonably small. Comparing across the three measurement models, factor analysis seems to be least appealing. Firstly, looking at the factor loading results shown in Table 5, the ten survey items return 4 factors with ss loading values larger than 1. Checking factor loadings associated with each item and their uniqueness parameters, we see a serious problem raised by intraitem heterogeneity. Not surprisingly, the assumption of local independence is violated when using factor analysis to predict the networking measure only based on the rst factor. Figure 5 further illustrates the problem of local dependence, by plotting all the other 4 factors against factor1 based on the specic factor loading scores associated with each networking item. Secondly, the factor-analytic model is very sensitive to the number of estimated factors. Using the 10 survey items, we nd that the estimation results of the rst factor would change substantially when dropping the number of predicted factors from 5 to 4. According to the results in Table 5, when estimating 5 factors with the 10 networking item, Local Hospital Ocials, Civic Group and State Hospital Association are not loaded on Factor1. When estimating 4 factors, however, State Hospital Association is the only item that does not load on Factor 1. Factor loading values for each item, furthermore, change substantially from the results in Table 5. For example, the factor loading for State Regulatory Agency would change from 0.345 to 0.378, and factor loadings for both StateM edicaid and StateM edicare would drop below 0.3. [Figure 5 About Here]
Social Service Agency). The sub-gure on the right-hand side of Figure 4 illustrates the diculty parameter specic to item 4 6, and choice 2. Because we dene j to be truncated by 0, simulated values for 46 are always positive. 5 In Table 5, factor analysis is performed using the f actanal() in the R package stats and the non-Bayesian IRT scaling is estimated using the gpcm() in the R package, ltm(Rizopoulos, 2006).

11

The latent dimension of managerial networking, in addition, can be inferred from the posterior distribution of i (see Figure 6). Our Bayesian analysis returns the values for i with a mean of 0.0 94 and a standard deviation of 9.975. The median value of the managerial networking scales is 0.037. Comparing the distribution plot of i estimated based on the Bayesian model and the IIC (Item Information Curve ) as well as the Test Information Function based on the non-Bayesian IRT analysis, we see that our Bayesian measure reduces the change in measurement scores dependent to the survey items. [Figure 6 About Here] Intra-Item/ Intra-Choice Compatibility. Further comparing the Bayesian IRT and nonBayesian IRT scaling, we nd that the two measurement approaches report comparable results, but the Bayesian IRT approach is superior with respect to reducing the cross item/cross choice compatibility issue. Our survey data show some cross item/choice compatibility issues. A few networking items have very skewed distributions toward the low-frequency end. The last choice category, Daily, is somehow problematic, because very a few respondents checked this category for some networking items. For example, based on our survey sample, less than 1% hospital managers interact with local/state public ocials and state regulatory agency on daily basis. The issues of intra-item and intra-choice compatibility leads to less reliable parameter estimation when tting a non-Bayesian generalized particle credit model. Figure 7 and 8 illiterates item information curve (IIC) and item characteristic curve (ICC) by each choice category. Both plots are generated after tting the GPCM reported in Table 5. Overall, the non-Bayesian IRT model recovers 86.6% information in the estimated latent dimension of networking. The estimated scale, however, is skewed toward the high-end (see subgure (2-2), the Test Information Curve in Figure 7 ). The estimated latent scale recovers 65.81% information for values between 0 and 4 (high level of networking), and only 20.8% information is recovered for values between -4 and 0 (low-level of networking). Comparing the by-item and overall IIC with the posterior mean distribution inferred from the Bayesian IRT model, the measurement scale returned by the Bayesian analysis covers to a normal distribution, whereby comparable information is measured for values below and above 0. The non-Bayesian IRT scale is also aected by cross-item heterogeneity with respect to the discriminative power of each item. Item 1 (Local Political Ocials) and Item 9 (State Hospital Association) has the highest discriminative power, with limited amount of information, these two items seem to distinguish between managers level of networking. Item 3 (State Regulatory Agency), 4(State Medicaid ), 5(State Medicare), and 8 (Insurance Companies) distinguish less among managers level of networking. Comparing the ranking of item-specic discriminative power with the discriminative parameters estimated by the Bayesian model, the Bayesian and non-Bayesian IRT produce comparable results on item 1 and 8, but produce slightly dierent results on item 1, 4, 5, and 8. Survey item 1,4,5,and 8 are all highly skewed toward low level of networking and the number of observations are quite limited on the high-networking values.The Bayesian IRT model improves performance of the 12

non-Bayesian IRT model regarding the information learned from these items via MCMC simulations. Last, but not least, Figure 8 further illustrates measurement issues caused by crosscategory compatibility. The ordinal survey responses are arbitrarily divided into 6 choice categories. Although the overall ranking is consistent across all items, respondents may not agree on the relative rankings among the 6 choices. Figure 8 shows a clear pattern of limited intra-item agreement on the 4 middle choice categories. Item 4 and 5 (State Medicare &Medicaid ) seem to stand separately from all the other items and these two items t with each other very well. Item 3 (State Regulatory Agency) stands closer to item 4 and 5 and seems to be distinguishable from all the other 7 networking nodes. Also our Bayesian model does not completely clear this problem, the intra-choice heterogeneity is reduced from the non-Bayesian IRT to a full-Bayesian IRT model if we compare j,k and the outputs of the IRT model (in Table 5).

Discussion

Our primary interest is to provide an alternative to the factor analytic techniques now common in public administration research. Factor analytic techniques provide powerful tools for addressing the measurement problems present in key areas of public administration research but bring with them a number of constraining assumptions (often ignored or otherwise concealed from readers). As an alternative, we propose a Bayesian IRT approach to measurement where these assumptions are transparent and chosen by the researcher rather than imposed by the technique. The requirement of specifying assumptions is particularly useful where measure theory and conceptions of dimensionality are limited in the literature. The unsettled nature of measurement in relation to collaborative networking is but one example. One could also nd use for this approach in other areas with unsettled measurement such as red tape, public service motivation, and organizational performance. The Bayesian approach to measurement is not, though, a panacea. The exibility of Bayesian IRT models allow researchers to develop poorly suited models even as they allow researchers to use more appropriate models. Furthermore, Bayesian IRT models can not themselves overcome the limitations of the input data. If the input data includes signicant missing data or a small sample (limiting statistical power), strong skew (reducing variation and again limiting statistical power), or unreliable questions (resulting in either high variance or systematic bias in the responses), the Bayesian IRT approach will only allow one to incorporate these problems but will not produce simple or useful measures. The advantage in these cases is that Bayesian IRT makes these problems transparent rather than concealing their presence behind the assumption that resulting factors will always have a mean zero and a standard deviation of one. Of course, this approach is only the beginning of the use of Bayesian IRT in public 13

administration research. We anticipate extending this analysis to including a discussion of missing data in measurement models and the intersection of imputation and IRT models. Furthermore, we would like to compare the measurement of external networking with the allocation of time connecting to internal nodes to assess the degree (or existence) of a tradeo between networking and internal management. Finally, we would like to compare various measurement approaches in the context of performance models. Does one measurement approach create measures that better explain organizational performance? What does this tell us about the nature of organizational performance and the survey reporting of such performance. The eld of public administration has not paid a great deal of attention to issues of measurement. With the proliferation of quantitative research and its extension into areas of study where existing, validated measures do not exist in other elds, it is important for public administration researchers to take measurement seriously. As this report illustrates, Bayesian IRT approaches provide an alternative to carefully assess measures and create what may turn out to be more eective elements for future research.

14

Table 1: Logistic Regression Predicting Survey Responses (Dependent Variable: Response=1, No-Response=0) Variable Coecient (Std. Err.) Service Type -0.005 (0.002) State 0.013 (0.017) Region -0.069 (0.164) Public Ownership 0.268 (0.088) Organizational Size 0.0003 (0.00005) Intercept -1.773 (0.128) N Log-likelihood 2 (5)
Signicance levels : : 10%

6351 -2752.537 95.638

: 5% : 1%

15

Table 2: Survey Response Rates by Organizational Type Owned By AHA Code No Response Response Total State 12 278 57 335 County 13 300 90 390 City 14 75 31 106 City-County 15 28 5 33 Hospital District 16 409 139 548 State/Local Govt. Overall 1090 322 1412 Church 21 480 75 555 Other 23 2182 408 2590 Non-Prot Overall 2661 483 3144 Investor 30 2 0 2 Individual 31 21 5 26 Partnership 32 226 50 276 Corporation 33 1137 140 1277 Private Overall 1386 195 1581 Air Force 41 9 1 10 Army 42 21 2 23 Navy 43 11 2 13 Public Health Service (PHS) 44 9 0 9 Veteran Service 45 120 15 135 PHS Indian Service 46 21 1 22 DOJ 48 2 0 2 Federal Overall 193 21 214 Unidentied NA NA 4 4 NA Total 5331 1025 6356

Rate(%) 17.01 23.08 29.25 15.15 25.36 22.80 13.51 15.76 15.36 0.00 19.23 18.12 10.96 12.33 10.00 8.70 15.38 0.00 11.11 4.55 0.00 9.81 NA 16.13

16

Table 3: Summary Statistics of All Networking Nodes (Listed by the tions) Networking Node Obs. Mean SD Internal Node Other Hospital Sta 1006 5.27 1.15 Hospital Corporate Oce 614 4.74 1.18 Medical Sta 1026 4.69 1.30 Patients 1009 4.82 1.10 Hospital Executive Committee 914 4.01 1.20 Hospital Dept. Management Group 983 4.09 1.20 External Node Hospital Board of Trustees 959 3.42 0.76 Volunteer Board 582 2.98 0.88 Hospital Foundation 578 3.13 0.76 Local Public Ocials 1000 3.05 0.85 Civic Groups 985 3.16 0.92 State Hospital Association 992 3.20 0.93 State Public Ocials 994 2.52 0.81 State Regulatory Agencies 991 2.39 0.73 State Medicaid 912 2.09 1.03 State Medicare 913 2.05 1.05 Accrediting 946 2.17 0.64 Social Service Agencies 911 2.33 1.08 Insurance Companies 949 2.40 1.19 IT vendors 912 2.19 1.12 Other Vendors 913 2.47 1.12 National Hospital Association 967 2.43 1.00 Other Professional Associations 905 2.69 0.92 Pharmaceutical Companies 880 1.64 1.10 Union Representatives 586 1.71 1.22

Frequency of InteracMedian 6 5 5 5 4 4 3 3 3 3 3 3 2 2 2 2 2 2 2 2 2 2 2 1 1

17

Table 4: The Discrimination (j ) and Diculty Parameter (j,k ): Summary of Posterior Distributions Based on the Generalized Partial Credit Model
j

Networking Node Local Political Ocials State Political Ocials State Regulatory Agency State Medicaid State Medicare Social Service Agency Civic Groups Insurance Company State Hospital Association National Hospital Association j,k (Posterior Mean) Networking Node Local Political Ocials State Political Ocials State Regulatory Agency State Medicaid State Medicare Social Service Agency Civic Groups Insurance Company State Hospital Association National Hospital Association

Mean 1.064 0.901 0.961 1.001 1.03 0.964 0.864 0.781 1.017 0.698 [j,2] Yearly -2.328 -2.875 -3.287 -0.522 -0.228 -0.431 -2.137 -0.486 -2.230 -2.050

Median 1.004 0.833 0.887 0.938 0.958 0.890 0.810 0.720 0.956 0.635 [j,3] Monthly -3.362 -2.589 -2.433 0.622 0.807 -0.049 -3.484 -0.170 -3.574 -1.220

SD 0.346 0.332 0.351 0.354 0.335 0.339 0.299 0.296 0.341 0.276 [j,4] Weekly -2.416 0.205 0.292 2.590 2.360 1.457 -2.901 1.859 -2.804 0.871

Naive SE 0.003 0.003 0.004 0.004 0.003 0.003 0.003 0.003 0.003 0.003 [j,5] > Weekly -0.733 0.800 0.485 3.176 3.458 3.170 -0.801 3.000 -1.478 2.151

TS-Se 0.027 0.023 0.029 0.016 0.012 0.010 0.023 0.010 0.028 0.012 [j,6] Daily 2.013 3.304 2.830 3.078 3.162 3.815 1.235 3.649 -0.076 3.781

18

Table 5: Managerial Networking: Factor Analysis and Non-Bayesian IRT alized Partial Credit Model) Networking Node Factor 1 Factor 2 Factor 3 Factor 4 Factor Analysis Local Political Ocials 0.216 0.114 0.954 State Political Ocials 0.155 0.621 0.151 0.321 State Regulatory Agency 0.345 0.776 0.123 0.117 State Medicaid 0.699 0.393 0.180 State Medicare 0.917 0.168 0.143 Social Service Agency 0.448 0.180 0.171 Civic Groups 0.125 0.195 0.406 Insurance Companies 0.599 0.127 0.112 State Hospital Association 0.932 0.146 National Hospital Association 0.282 0.219 0.526 0.128 SS Loadings 2.133 1.336 1.304 1.275 Proportion Var 0.213 0.134 0.130 0.128 Cumulative Var 0.213 0.347 0.477 0.605 Ordinal IRT Coecients Cat.1 Cat.2 Cat.3 Cat.4 Local Political Ocials -4.587 -1.916 1.706 3.417 State Political Ocial -3.260 0.252 3.178 1.934 State Regulatory Agency -2.828 0.628 2.601 1.891 State Medicaid -0.610 0.892 1.953 2.043 State Medicare -0.455 0.979 1.938 2.348 Social Service Agency -0.840 0.547 2.317 3.210 Civic Groups -4.771 -2.733 1.173 4.352 Insurance Companies -0.814 0.355 2.314 2.148 State Hospital Association -5.468 -2.750 1.571 3.072 National Hospital Association -2.437 0.836 2.564 2.303

Analysis (GenerFactor 5 0.118 0.102 0.113 0.547 0.277 0.253 0.143 0.499 0.050 0.655 Cat.5 5.822 4.681 4.099 2.364 2.186 2.780 4.704 2.388 3.694 3.398 Uniq. 0.022 0.451 0.240 0.305 0.103 0.434 0.699 0.544 0.096 0.559

Discr. 0.612 0.878 1.384 2.371 1.730 0.806 0.423 0.828 0.475 0.623

19

Figure 1: An Illustration of Ordinal Subject-Item Response Matrix

Frequency 100 150 200 250 300 350 400 0 50

Subject 1 2 3 4 5 6 7 n

Item1 6 2 6 3 6 6 6 6

3 Item 1

Subject 1 2 3 4 5 6 7 n Subject 1 2 3 4 5 6 7 n

Item 2 5 2 5 5 5 5 1 4 Item j 3 2 3 3 3 1 1 4

50

Frequency 100 150 200 250 300 350 400

1
600

3 Item 2

100

200

Frequency 300 400

500

3 Item j

20

Figure 2: An Illustration of the Association between an Item Pair

Item 2

1
1

3 Item 1

21

350

400

300

300

300

250

300

250

200

200

200

200

Frequency

Frequency

150

Frequency

Frequency

150

Frequency

100

100

100

100

50

50

1 State Public Officials State Regulatory Agencies

50

100

150

200

250

300

4 State Medicare

Local Public Officials

State Medicaid

250

300

350

250

200

300

200

200

250

150

150

150

200

Frequency

Frequency

Frequency

100

Frequency

150

100

100

Frequency 100 0 50

50

50

50

1 Civic Groups

5 Insurance Companies

5 State Hospital Association

50

100

150

200

250

300

Figure 3: Frequencies of Managerial Networking with Ten External Partners (1=Never...6=Daily)

22
1 2 3 4 5 National Hospital Association 6

Social Service Agency

Figure 4: Trace Plots of Selected Parameters

Trace of alpha[4]
3.5 1.2

Density of alpha[4]

Trace of beta[4,3]
1.5

Density of beta[4,3]

0.5

0.0

0.5

0e+00 2e+04 4e+04 6e+04 8e+04 1e+05 Iterations

0e+00 2e+04 4e+04 6e+04 8e+04 1e+05 Iterations

0.0

0.0

0.5

1.0

1.5

N = 10000 Bandwidth = 0.05315

N = 10000 Bandwidth = 0.03749

Trace of alpha[5]
3.0 1.2

Density of alpha[5]
2.0

Trace of beta[5,3]
1.5

Density of beta[5,3]

0.5

0.0

0.5

0e+00 2e+04 4e+04 6e+04 8e+04 1e+05 Iterations

0.5

1.0

1.5

2.0

2.5

3.0

0e+00 2e+04 4e+04 6e+04 8e+04 1e+05 Iterations

0.0

0.0

0.5

1.0

1.5

2.0

N = 10000 Bandwidth = 0.05224

N = 10000 Bandwidth = 0.04376

Trace of alpha[6]
3.0 1.5

Density of alpha[6]

Trace of beta[6,3]
3.0

Density of beta[6,3]

0.5

0.0

-0.5

0e+00 2e+04 4e+04 6e+04 8e+04 1e+05 Iterations

0.5 1.0 1.5 2.0 2.5 3.0 3.5 N = 10000 Bandwidth = 0.0486

0e+00 2e+04 4e+04 6e+04 8e+04 1e+05 Iterations

0.0

-0.5

0.0

0.5

N = 10000 Bandwidth = 0.01936

23

Figure 5: Maximum Likelihood Factor Analysis: Comparison of the First Five Factors

0.7

Factor 2

Factor 3

0.5

0.3

0.1

0.2

0.4

0.6

0.8

0.2

0.4

0.6

0.8

0.2

0.4

0.6

0.8

Factor 1

Factor 1

0.8

Factor 4

0.4

Factor 5

0.6

0.2

0.2

0.4

0.6

0.8

0.0

0.2

0.4

0.2

0.4

0.6

0.8

Factor 1

Factor 1

24

Figure 6: MCMC Sample of i : Trace Plot, Autocorrelation Plot, and Density Plot

20

-20

0e+00

4e+04 Iterations

8e+04

0.00

-40

0.02

0.04

40

-40

-20

20

40

N = 10000 Bandwidth = 1.677

Autocorrelation

-1.0
0

-0.5

0.0

0.5

1.0

100

200 Lag

300

400

25

Figure 7: The Item Information Characteristic Curves based on the non-Bayesian Partial Credit Model (V1= Local Political Ocials, V2=State Political Ocials, V3=State Regulatory Agency,
V4=State Medicaid, V5=State Medicare, V6=Social Service Agency, V7=Civic Groups, V8=Insurance Companies, V9=State Hospital Association, V10=National Hospital Association)
Item Information Curves
V1 V2 V3

Item Information Curves 6

V4 V5 V6

Information

1.5

Information -4 -2 0 2 4

1.0

0.5

0 -4

-2

Latent Dimension of Networking

Latent Dimension of Networking

Item Information Curves 1.0

V7 V8 V9 V10

Test Information Function

Information

0.8

Information -4 -2 0 2 4

0.6

0.4

0.2

2 -4

10

14

-2

Latent Dimension of Networking

Latent Dimension of Networking

26

Figure 8: The Item Category Characteristic Curves based on the non-Bayesian Partial Credit Model (V1= Local Political Ocials, V2=State Political Ocials, V3=State Regulatory Agency,
V4=State Medicaid, V5=State Medicare, V6=Social Service Agency, V7=Civic Groups, V8=Insurance Companies, V9=State Hospital Association, V10=National Hospital Association, Category 1= Never, Category 2=Yearly, Category 3=Monthly, Category 4= Weekly, Category 5=More than Weekly, Category 6=Daily )
Item Response Category Characteristic Curves Category: 1
1.0 1.0

Item Response Category Characteristic Curves Category: 2

1.0
Probability

Item Response Category Characteristic Curves Category: 3

0.8

0.8

V1 V2 V3 V4 V5 V6 V7 V8 V9 V10

0.6

0.6

Probability

0.4

Probability

0.4

0.2

0.2

0.0

0.0

-4

-2

-4

-2

0.0

0.2

0.4

0.6

0.8

-4

-2

Latent Dimension of Networking

Latent Dimension of Networking

Latent Dimension of Networking

Item Response Category Characteristic Curves Category: 4

1.0 1.0

Item Response Category Characteristic Curves Category: 5

1.0
Probability

Item Response Category Characteristic Curves Category: 6

0.8

0.8

0.6

0.6

Probability

0.4

Probability

0.4

0.2

0.2

0.0

0.0

-4

-2

-4

-2

0.0

0.2

0.4

0.6

0.8

-4

-2

Latent Dimension of Networking

Latent Dimension of Networking

Latent Dimension of Networking

27

References
Agrano, Robert and Michael McGuire. 1999. Managing in Network Settings. Public Administration Review 16(1):1841. Agrano, Robert and Michael McGuire. 2001. Big Questions in Public Management Research. Journal of Public Administration Research and Theory 11(3):295326. Agrano, Robert and Michael McGuire. 2003. Collaborative Public Management: New Strategies for Local Government. Washington, D.C.: Georgetown University Press. Akkerman, Agnes and Ren Torenvlied. 2011. Managing the Environment. Public Admine istration Review 13(1):15974. Anderson, Erling B. 1997. The Rating Scale Model. In Handbook of Modern Item Response Theory, ed. Wim J. van der Linden and Ronald K. Hambleton. New York: Springer pp. 6784. Andrew, Rhys, George A. Boyne, Kenneth J. Meier, Laurence J. OToole and Richard M. Walker. 2011. Environmental and Organizational Determinants of External Networking. The American Review of Public Administration 41(4):35574. Bafumi, Joseph, Andrew Gelman, David K. Park and Noah Kaplan. 2005. Practical Issues in Implementing and Understanding Bayesian Ideal Point Estimation. Political Analysis 13(2):17187. Bradlow, Eric T. and Alan M. Zaslavsky. 1999. A Hierarchical Latent Variable Model for Ordinal Data from a Customer Satisfaction Survey with No Answer Reponse. Journal of American Statistical Association 94(445):4352. Brady, Henry E. 1985. The Perils of Survey Research: Inter-Personally Incomparable Responses. Political Methodology 11(3/4):269291. Curtis, S. McKay. 2010. BUGS Code for Item Response Theory. Journal of Statistical Software 36(1):Code Snippet 1. Donaldson, Stewart I. and Elisa J. Grant-Vallone. 2002. Understanding Self-Report Bias in Organizational Behavior Research. Journal of Business and Psychology 17(2):245260. Doty, D. Harold and William H. Glick. 1998. Common Methods Bias: Does Common Methods Variance Really Bias Results? Organizational Research Methods 1(4):374406. Embretson, Susan E. and Steven P. Reise. 2000. Item Response Theory for Psychoogists. Mahwah, New Jersey: Lawrence Erlbaum Association, Inc. Fox, Jean-Paul. 2005. Randomize Item Response Theory Models. Journal of Educational and Behavioral Statistics 30(2):189212. 28

Fox, Jean-Paul. 2010. Bayesian Item Response Modeling: Theory and Applications. New York: Springer. Fumiko, Samejima. 2008. Graded Response Model Based on the Logistic Positive Exponent Family of Models for Dichotomous Responses. Psychometrika 73(4):56178. Geletkanycz, Marta A., Brian K. Boyd and Sidney Finkelstein. 2001. The Strategic Value of CEO External Directorate Networks: Implications for CEO Compensation. Strategic Management Journal 22(9):88998. Graham, John W. and Nancy L. Collins. 1991. Controlling Correlational Bias via Conrmatory Factor Analysis of MTMM. Multivariate Behavioral Research 26(4):607629. Hemker, Bas T., Klass Sijtsma and Ivo W. Molenaar. 1995. Selection of Unidimensional Scales From a Multidimensional Item Bank in Polytomous Mokken IRT Model. Applied Psychological Measurement 19(4):337352. Henry, Adam Douglas, Mark Lubell and Michael McCoy. 2012. Survey-Based Measurement of Public Management and Policy Networks. Journal of Policy Analysis and Management 31(2):43352. Hicklin, Alisa K., Kenneth J. Meier and Jr. OToole, Laurence J. 2008. Serpents in the Sand: Managerial Networking and Nonlinear Inuences on Organizational Performance. Journal of Public Administration Research and Theory 18(2):25374. Isett, Kimberley R., Ines A. Mergel, Kelly LeRoux, Pamela A. Mischen and R. Karl Rethemeyer. 2011. Networks in Public Administration Scholarship: Understanding Where We Are and Where We Need to Go. Journal of Public Administration Research and Theory 21(suppl.):i157i173. Jackman, Simon. 2009. Bayesian Analysis for the Social Science. West Sussex, UK: John Wiley and Sons, Ltd. King, Gary, Christopher J.L. Murry, Joshua A. Salomon and Ajay Tandon. 2004. Enhancing the Validity and Cross-Cultural Comparability of Measurement in Survey Research. American Political Science Review 98(1):191207. King, Gary and Jonathan Wand. 2007. Comparing Incomparable Survey Responses: New Tools for Anchoring Vignettes. Political Analysis 15(1):4666. Masters, Geoggerey N. 1988. Analysis of Partial Credit Scoring. Applied Measurement in Education 1(4):279297. McGuire, Michael. 2001. Managing Networks: Propositions on What Managers Do and Why They Do It. Public Adminsitration Review 62(5):599609.

29

Meier, Kenneth J. and Jr. OToole, Laurence J. 2001. Managerial Stratigies and Bebaviors in Networks: A Meodel with Evidence from U.S. Public Education. Journal of Public Administration Research and Theory 11(3):27194. Meier, Kenneth J. and Jr. OToole, Laurence J. 2005. Managerial Networking: Issues of Measurement and Research Design. Adminsitration and Society 37(4):52341. Meier, Kenneth J. and Jr. OToole, Laurnece J. 2010. Organizational Performance: Measurement Theory and An Application: Or, Common Source Bias, the Achilles Heel of Public Management Research. APSA 2010 Annual Meeting Paper. Available at SSRN: http://ssrn.com/abstract=1642740. Mokken, Robert J. 1971. A Theory and Procedure of Scale Analysis with Applications in Political Research. New York: De Gruyte. Ostini, Remo and Michael L. Nering. 2006. Polytomous Item Response Theory Models. Thousand Oaks, CA: Sage Publication. OToole, Laurence J., Jr., Alisa Hicklin, Scott Robinson and Kenneth J. Meier. 2006. Calming the Storms: Collaborative Public Management, Hurrican Katrina and Rita, and Disaster Responses. Presented at the Research Conference on Collaborative Public Management, Washington D.C.,September 28-30. OToole, Laurence J., Jr. and Kenneth J. Meier. 2011. Public Management: Organizations, Governance, and Performance. New York: Cambridge University Press. Podsako, Philip M. and Dennis W. Organ. 1986. Self-Reports in Organizational Research: Problems and Prospects. Journal of Management 12(4):533543. Rizopoulos, Dimitris. 2006. ltm: An R Package for Latent Variable Modeling and Item Response Theory Analysis. Journal of Statistical Software 17(5):125. Robinson, Scott. 2006. A Decade of Treating Networks Seriously. Policy Studies Journal 34(4):58998. Samijima, Fumiko. 1969. Estimation of Latent Ability Using a Response Pattern of Graded Scores. Psychometrika Monograph Supplement p. No.17. Schwarz, Norbert. 1999. Self-Reports: How the Questions Shapre the Answers. American Psychologist 54(2):93105. Thomson, Ann M., James L. Perry and Theodore K. Miller. 2009. Conceptualizing and Measuring Collaboration. Journal of Public Administration Research and Theory 19(1):2356. Torenvlied, Ren, , Agnes Akkerman, Kenneth J. Meier and Laurence J. OToole. e 2012. Multiple Dimensions of Managerial Networking. Advanced access, DOI: 10.1177/0275074012440497. 30

Torenvlied, Ren and Agnes Akkerman. 2012. Eects of Managers Working Motivation and e Networking Activity on Their Reported Levels of External Red Tape. Online Advanced Access, February 29. Treier, Shawn and Simon Jackman. 2008. Democracy as a Latent Variable. American journal of Political Science 52(1):20117. van der Linden, Wim J. and Ronald K. Hambelton. 1996. Item Response Theory: Brief History, Common Models, and Extensions. In Handbook of Modern Item Response Theory, ed. Wim J. van der Linden and Ronald K. Hambelton. New York: Springer pp. 128. van Schuur, Wijbrandt H. 2003. Mokken Scale Analysis: Between the Guttman Scale and Parametric Item Response Theory. Political Analysis 11(2):139163. van Schuur, Wijbrandt H. 2011. Ordinal Item Response Theory: Mokken Scale Analysis. Thousand Oaks, CA: Sage Publication.

31