Psycho Metrics 08

Questionnaires and measures validation: Psychometric models: Latent State-Trait models and Multitrait-Multimethod models
Delphine Courvoisier, Olivier Renaud Section of Psychology University of Geneva
Delphine Courvoisier & Olivier Renaud
Psychometric models summer 07 1
General Information
Introduction Information Materials Exam Bibliography Psychometrics SEM: Theory SEM: Examples LST: Theory LST: Example MTMM: Theory MTMM: Examples MC Study: Theory MC Study: Example
Optional course (3 ECTS) with practicals included. Teachers: Delphine Courvoisier (delphine.courvoisier@cisa.unige.ch) Olivier Renaud (Olivier.Renaud@pse.unige.ch) Time: tuesday 16h15 18h Place: UniMail M2160 / M5183 Website: https://dokeos.unige.ch/home/courses/751514 Work load: 3 ECTS = 6 work hours per week: course: 2 hours practicals with Mplus: 2 hours home reading (6 articles): 2 hours
Delphine Courvoisier & Olivier Renaud Psychometric models summer 07 2
Materials for course and practicals

Dokeos. Every document used in this course and practicals can be found on the centralised website called Dokeos. You should download and print these document because they will not be distributed during the course. If you do not have a computer, the faculty has several computer rooms with internet connexion. The documents on Dokeos are very summarized and therefore insucient to understand the course and practicals. Materials should be available at the latest the week-end before the course.
Exam
The exam will be in july (or september). Grades range from 0 to 6 (4=sucient). They will be based on: 10%: Summary of your analysis of a data set: prepared before the written exam. 90%: Written exam (2 hours) with questions on: Theory Interpretation of the output of your analysis For the extraordinary exam session (january-february), the evaluation will be an oral exam.
Bibliography: Validity
Borsboom, D., Mellenbergh, G. J., van Heerden, J. (2004). The concept of validity. Psychological Methods, 4, 1061 1071. Schmitt, M. (2006). Conceptual, theoretical, and historical foundations of multimethod assessment. In M. Eid, E. Diener (Eds.) Handbook of multimethod measurement in psychology. Washington, DC: American Psychological Association (pp. 9 25).
Bibliography: Structural Equation Models

Bacher, F. (1987). Les mod`les structuraux en psychologie. e Prsentation dun mod`le Lisrel. Premi`re partie. Le Travail e e e Humain, 50, 347370. Bacher, F. (1987). Les mod`les structuraux en psychologie. e Prsentation dun mod`le Lisrel. Deuxi`me partie. Le Travail e e e Humain, 51, 273288. Hoyle, R. H. (1995). The structural equation modeling approach: Basic concepts and fundamental issues. In R. H. Hoyle (Ed.), Structural equation modeling (pp. 115). Thousands Oaks, CA: Sage Publications. Schermelleh-Engel, K., Moosbrugger, H., Mller, H. (2003). u Evaluating the t of structural equation models: tests of signicance and descriptive goodness-of-t measures. Methods of Psychological Research Online, 8(2), 23-74.
Bibliography: Latent State Trait theory and models

Steyer, R., Ferring, D., Schmitt, M. J. (1992). States and traits in psychological assessment. European Journal of Psychological Assessment, 8(2), 79-98. Steyer, R. ; Schmitt, M. & Eid, M. (1999). Latent state-trait theory and research in personality and individual dierences. European Journal of Personality, 13, 389-408. Courvoisier, D. S. (2006). Unfolding the constituents of psychological scores: Development and application of mixture and multitrait-multimethod LST models. Unpublished doctoral dissertation, University of Geneva, Switzerland. (pp. 1423).
Bibliography: Multitrait Multimethod theory and models

Eid, M. ; Lischetzke, T. ; Nubeck, F. & Trierweiler, L. (2003). Separating trait eects from trait-specic method eects in multitrait-multimethod analysis: A multiple indicator CTC(M1) model. Psychological Methods, 8, 38-60. Eid, M. (2000). A multitrait-multimethod model with minimal assumptions. Psychometrika, 65, 241-261. Eid. M, Lischetzke, T., Nussbeck, F. W. (2006). Structural equation models for multitrait-multimethod data. In M. Eid, E. Diener (Eds.) Handbook of multimethod measurement in psychology. Washington, DC: American Psychological Association (pp. 283299). Courvoisier, D. S. (2006). Unfolding the constituents of psychological scores: Development and application of mixture and multitrait-multimethod LST models. Unpublished doctoral dissertation, University of Geneva, Switzerland. (pp. 6269).
Bibliography: Monte Carlo studies

Muthn, L. K., Muthn, B. O. (2002). How to use a monte e e carlo study to decide on sample size and determine power. Structural Equation Modeling, 9, 599620. Courvoisier, D. S., Eid, M. & Nussbeck, F. W. (2007). Mixture Distribution Latent State-Trait Analysis: Basic Ideas and Applications. Psychological Methods, 12(1), 80104.
Validity: denition
Introduction Psychometrics Validity: Def. Validities Reliability: Def. Reliabilities Visually Validation: Why Validation: How Psychometrics:Def Classical test Error Measures Inuences SEM: Theory SEM: Examples LST: Theory LST: Example MTMM: Theory MTMM: Examples MC Study: Theory MC Study: Example Delphine Courvoisier & Olivier Renaud Psychometric models summer 07 10
A test is valid if it measures what it purports to measure (Kelley, 1927, p. 14) Do the empirical relations between test scores match theoretical relations in a nomological network (Cronbach & Meehl, 1955) Are interpretations and actions based on test scores justied not only in the light of scientic evidence but with respect to social and ethical consequences of test use? (Messick, 1989)
Validity: denition
Introduction Psychometrics Validity: Def. Validities Reliability: Def. Reliabilities Visually Validation: Why Validation: How Psychometrics:Def Classical test Error Measures Inuences SEM: Theory SEM: Examples LST: Theory LST: Example MTMM: Theory MTMM: Examples MC Study: Theory MC Study: Example Delphine Courvoisier & Olivier Renaud Psychometric models summer 07 11 Schmitt, M. (2006)
A test is valid for measuring an attribute if and only if a) the attribute exists and b) variations in the attribute causally produce variations in the outcomes of the measurement procedure (Borsboom, Mellenbergh & van Heerden, 2004). Correlations cannot provide more than circumstantial evidence for validity. Problem of validity cannot be solved by psychometric techniques or models alone. (Borsboom, et al., 2004)
Validities
Internal validity: Certainty with which a researcher can say that observed changes function of the conditions of treatment of the experience were really caused by the independent variable (Myers & Hansen, 2003, p. 101). External validity: Quality of an experiment results that can be generalized or applied to other subjects and other situations that were not directly tested. (Myers & Hansen, 2003, p. 101). Convergent validity: Measures of the same construct measured by dierent methods should be similar. Discriminant validity: Measures of several constructs should dier. Incremental validity: Measures of a new construct should increase the explanatory power of other related variables.
Validities (2)
Introduction Psychometrics Validity: Def. Validities Reliability: Def. Reliabilities Visually Validation: Why Validation: How Psychometrics:Def Classical test Error Measures Inuences SEM: Theory SEM: Examples LST: Theory LST: Example MTMM: Theory MTMM: Examples MC Study: Theory MC Study: Example Borsboom, D. (2004)
Construct validity: How much are the measures used representative of the concepts and domains of our hypothesis? Is the relation observed between the independent and the dependent variables also correct at the construct level? Risks: Bad operationalization Not all modalities of the IV were measured Only one method was used to measure the construct Interaction between treatments (all things being equal) Face validity: Does the measure seem valid? and (too?) many more . . .
Reliability: denition
Constant: Measure stay constant every time it is assessed and on every object assessed. A reliable car starts every time the keys are turned. Coherent: Several measures are coherent. Researchers measuring the same subject with dierent instruments should obtain the same result. When using a reliable operational denition to measure a specic characteristic in similar groups, it should yield the same result. WARNING: some characteristics change across time!
Reliabilities
Interjudge. Used to assess the degree of similarity between several judges measuring the same phenomenon Test-retest. Used to assess the stability of a measure across time (inappropriate for changing phenomenon) Parallel forms. Used to assess the coherence of measures build in the same way with questions coming from the same set of questions. Internal coherence. Used to assess the coherence of items of the same measure. ...
Graphical representation of reliability and validity

Validation of questionnaire: Why?

Introduction Psychometrics Validity: Def. Validities Reliability: Def. Reliabilities Visually Validation: Why Validation: How Psychometrics:Def Classical test Error Measures Inuences SEM: Theory SEM: Examples LST: Theory LST: Example MTMM: Theory MTMM: Examples MC Study: Theory MC Study: Example
Measuring imprecisely Measuring precisely several underlying constructs To assess validities by way of reliabilities But reliability is still not equal to validity. Correlations cannot provide more than circumstantial evidence for validity. Problem of validity cannot be solved by psychometric techniques or models alone. (Borsboom, et al., 2004) + Estimate the dierent sources of inuence on the measure trait occasion of measurement experimenter method error of measurement ...
Validation of questionnaire: How?

Correlations: Interjudge: between judges Test-retest: between the two occasions of measurement Parallel forms: between the two forms Internal coherence: mean of correlations between each item and the sum of the other items OR correlation between two randomly created subtests (ex. odd vs. even items) Cronbachs
Structural Equations Models use the correlation (or covariance) matrix of all items
Psychometrics: denition
Denition: Science studying measurement techniques (in psychology but also in other sciences) as well as the validation techniques of these measures. Initially developped for measuring intellectual performances (mental ages, intellectual quotient, development quotient for young children, . . . ) and personality (aectivity, emotions, . . . ). An indicator that does not have good psychometric properties limits the interpretations of the results of an empirical research. The signal may be lost in the noise. Researchers must estimate the psychometric properties of their measures.
Classical test theory

We suppose that for each measure of an object Y , measurement error E stop us from obtaining the true score T of this object. Moreover, this error can have a random component er as well as a systematic component es : Y =T +E E = es + er The main goal of psychometrics is to avoid that the measurement error E be: unknown large systematic (non-random) related to the constructs studied
Graphical representation of measurement error

Random error (left) decrease precision of the results, and thus condence in the reesults. However, results are not biased.
Graphical representation of measurement error

Random error (left) decrease precision of the results, and thus condence in the reesults. However, results are not biased. On the contrary, systematic error (right) introduce bias in the results, thereby decreasing validity.
Inuences on measures
A psychological score Y
may vary as a function of:
A psychological score Yj
may vary as a function of:
Stable construct j
A psychological score Yjk may vary as a function of:
Stable construct j Method of measurement k
A psychological score Yjkl may vary as a function of:
Stable construct j Method of measurement k Momentary situations l
A psychological score Yjklc may vary as a function of:
Stable construct j Method of measurement k Momentary situations l Structural dierences c
A psychological score Y jklc + may vary as a function of:
Stable construct j Method of measurement k Momentary situations l Structural dierences c Measurement error
A psychological score Y jklc + may vary as a function of: LST Stable construct j Method of measurement k Momentary situations l Structural dierences c Measurement error X X X
A psychological score Y jklc + may vary as a function of: LST MTMM Stable construct j Method of measurement k Momentary situations l Structural dierences c Measurement error X X X X X X
A psychological score Y jklc + may vary as a function of: LST MTMM Mixture Stable construct j Method of measurement k Momentary situations l Structural dierences c Measurement error X X X X X X X
A psychological score Y jklc + may vary as a function of: LST MTMM Mixture Stable construct j Method of measurement k Momentary situations l Structural dierences c Measurement error X X X X X X X
To assess variation, one must measure a construct several times while varying the source(s) of inuence of interest.
A psychological score Yjklc + may vary as a function of: mixture LST Stable construct j Method of measurement k Momentary situations l Structural dierences c Measurement error X X X X
A psychological score Yjklc + may vary as a function of: mixture LST MTMM-LST Stable construct j Method of measurement k Momentary situations l Structural dierences c Measurement error X X X X X X X X
A psychological score Yjklc + may vary as a function of: mixture LST MTMM-LST Stable construct j Method of measurement k Momentary situations l Structural dierences c Measurement error X X X X X X X X
Theoretically, each set of sources of inuence can be modeled.
Sources of inuence
Traits (ex. depression, well-being) Response modes (ex. motor, verbal, physiological) Traits and response modes may overlap (ex. cognitive, conative and emotional response modes to depression could be three subtraits of depression) Dimensions of measurement (ex. frequency, number of errors) Settings of measurement (ex. classroom, home, lab) Sources of measurement (ex. self-rating, friend rating) Instruments of measurement (ex. implicit or explicit measures) Methods of measurement (ex. interview, questionnaire, behavioral observation) Occasions of measurement ...
Conrmatory factor analysis (CFA) and structural equation models (SEM)

Introduction Psychometrics SEM: Theory CFA-SEM Denition Principles H0 Goodness-of-t Steps Submodels Scaling Missing info Popularity Diagram SEM: Examples LST: Theory LST: Example MTMM: Theory MTMM: Examples MC Study: Theory MC Study: Example Bacher, F. (1987) OR Hoyle, R.H. (1995)
CFA often follows an exploratory factor analysis (EFA). In a CFA, one species: number of factors non zero loadings correlations between factors SEM are a very large class of data analysis. For example, CFA is a specic type of SEM in which existing factors can only correlate with other factors.
SEM: denition
Introduction Psychometrics SEM: Theory CFA-SEM Denition Principles H0 Goodness-of-t Steps Submodels Scaling Missing info Popularity Diagram SEM: Examples LST: Theory LST: Example MTMM: Theory MTMM: Examples MC Study: Theory MC Study: Example
SEM are also called: Simultaneous equation modeling Covariance structure analysis Causal modeling Causal analysis LiSRel or EQS models Path analysis ...
Denitions : SEM are a formal description of the relations between specic observed and latent variables Hypothetical pattern of relations between variables Conrmatory techniques used to validate or falsify hypotheses about a specic analysis model.
Basic principles of SEM

SEM are specied from theoretical models that explain the data structure. The analysis is based on variances and covariances (or correlations) of the observed variables. Every SEM is a compromise between: precision: assessed by several goodness-of-t criteria. All these indices compare the observed covariance matrix with the predicted covariance matrix. parsimony: assessed by comparing the available information with the information necessary to estimate the model (p, the number of parameters). v(v + 1) degrees of freedom (df ) = p 2 where v = number of variables in S (the observed covariance matrix) and p = number of estimated parameters.
Null hypothesis in SEM

H 0 : = p So, researchers hope NOT to reject the null hypothesis. (p )!!! violation of assumptions: observed variables are supposed to be multivariate normal and sample size is supposed to be suciently large. model complexity: 2 decreases when p increases (not parsimonious) dependence on sample size: 2 increases with sample size. E(2 ) = df discrepance discrepance model data model reality coherence model reality good t model data ???? bad t
Goodness-of-t criteria
= 2: good t, = 3: acceptable t RMSEA: root mean square error of approximation < .05 good t, .05 .08 acceptable t SRMR: standardized root mean square residual < .05 good t, < .10 acceptable t Descriptive measures based on model comparisons: NFI, NNFI (TLI), CFI, GFI, AGFI > .97 good t, > .95 acceptable t Descriptive measures based on parsimony: PGFI, PNFI (> .97 good t, > .95 acceptable t) Badness of t: AIC, CAIC, ECVI (the best model has the smallest index)
Schermelleh-Engel, K. (2004)
2 df
2 df
Advised set of of indices

2 and its p
2 df
RMSEA and condence interval SRMR NNFI CFI
Steps to test SEM

1. choose variables of interest 2. explore characteristics and relations of chosen variables 3. propose a structure (specic and parsimonious) 4. draw the diagram of this structure 5. propose alternate models 6. draw the diagrams of the alternate models 7. test the prefered model as well as the alternate models 8. check the results 9. compare the goodness-of-t criteria Each SEM has equivalent models, i.e. models that have the same Delphine Courvoisier & Olivier Renaud Psychometric models summer 07 31 p but that dier on the theoretical level. One must take this into
Submodels of complex SEM

CFA are SEM in which factors can only correlate. Some SEM propose more complex relations between factors. For example, regression between factors. In complex SEM, one can distinguish two submodels: 1. Measurement model: to dene latent variables Similar to factor analysis 2. Structural model: to dene relations between latent variables Similar to either correlation or regression Best practice: test all measurement models. THEN test the structural models.
Factor identication in SEM (scaling)

Factors, being latent variables, are not scaled. It is therefore necessary to establish articially a scale. Two possibilities to identiy the factors : 1. x their variance (for example to 1). 2. x a loading coecient to 1. To identify the model, two conditions are necessary (but not sufcient): 1. each factor is identied 2. the degrees of freedom are not negative Moreover, these two conditions must be satised locally, that is for each measurement model and for the structural model, and globally.
Informations often missing in articles using SEM

1. Theoretical justication of proposed model 2. Denition or explanation of causality 3. Justication of the estimation method and proof of the validity of the underlying assumptions 4. Several goodness-of-t criteria 5. Theoretical and statistical justication of the modication of the model 6. Enough information to allow replication of the analyses
From Hershberger, S. L. (2003). The growth of structural equation modeling: 1994-2001. Structural Equation Modeling, 10, 35-46.
Hershberger, 2003 (suite)

Diagram elements
To understand and present models analyzing the data structure, it is necessary to use a precise graphical representation, a diagram. Observed variable
Latent variable
Asymetrical eect (regression weight) Symetrical eect (variance and covariance)
Simple regression
Introduction Psychometrics SEM: Theory SEM: Examples Simple regression Multiple regression CFA-2 simple AFC-2 complex Saturated Independence Equivalent M+ input: syntax Data M+ output LST: Theory LST: Example MTMM: Theory MTMM: Examples MC Study: Theory MC Study: Example
Vx
=1
Y = x + where Y and x are standardized (m = 0, s = 1) and of zero and a variance of 2 . has a mean
In this model, the value of Y for an individual i is Yi = xi + And the variance of Y is: V (Y ) = 2 V (x) + 2 2(2 + 1) dl = 3=0 2
Multiple regression
Vx1 Vx2
x1 x2
1 2
=1
Cx1x2
Y = 1 x1 + 2 x2 +
2 2 V (Y ) = 1 V (x1 ) + 2 V (x2 ) + 21 2 C(x1 ; x2 ) + 2
3(3 + 1) dl = 6=0 2
2 factors conrmatory factor analysis, simple structure

=1 f1
Cf1f2
=1 f2 5 6
1 2 3 4 x1
=1
x2
=1
x3
=1
x4
=1
x5
=1
x6
=1
u1 1 2
u2 2 2
u3 3 2
u4 42
u5 5 2
u6 6 2
dl = (6(6 + 1)/2) 13 = 8
2 2 V (x1 ) = 2 + 1 ; V (x5 ) = 2 + 5 1 5
C(x1 ; x5 ) = 1 C(f1 ; f2 )5
2 factors conrmatory factor analysis, complex structure

=1 f1
Cf1f2
=1 f2 5 6 7
1 2 3 4 x1
=1
x2
=1
x3
=1
x4
=1
x5
=1
x6
=1
u1 1 2
u2 2 2
u3 3 2
u4 42
u5 5 2
u6 6 2
dl = (6(6 + 1)/2) 14 = 7
2 V (x4 ) = 2 + 2 + 24 C(f1 ; f2 )5 + 4 4 5
C(x4 ; x5 ) = 4 C(f1 ; f2 )6 + 5 6
Saturated model
Saturated model
A B C D E F
All variables correlate and have a variance. 6(6 + 1) dl = 21 = 0 2
Independence model
Independence model
A B C D E F
Variables are uncorrelated and all variables have a variance. dl = 6(6 + 1) 6 = 15 2
Example of two equivalent SEM

A 1
B 1
E 2 2 1
1 A 1 B C D
Even though both models produce the same p , their theoretical interpretations are very dierent.
Questions: Psychometrics and SEM

Dene reliability Dene validation Dene validity How can we test validity? What is the dierence between using SEM to estimate a multiple regression and estimating a multiple regression with ordinary least square? What are the df of a model with 5 observed variables and one common factor. Draw the diagram and indicate all parameters. What is a saturated and an independent model?
Mplus input: syntax

By convention: LATENT VARIABLES IN CAPITALS (LV) observed variables in small letters (obs) In Mplus: Each line must end by ; and maximum length of 80 characters. TITLE: title text will be written in the rst lines of the output DATA: FILE IS species where is the le FORMAT: 33F8.2 means 33 variables taking 8 characters per variables (5 characters for integer, one dot and 2 decimals) TYPE IS individual for raw data VARIABLE gives list of variables names USEVARIABLES indicates which variables will be used in the analysis
Mplus input: syntax(2)

ANALYSIS: TYPE IS general meanstructure (estimate the covariances and the mean structure) ESTIMATOR: for example, maximum likelihood (ML), maximum likelihood robust (MLR) MODEL: begins description of the model. First loading is xed to 1 by Mplus even if written LV by obs1 OUTPUT: indicates which outputs should be printed. sampstat: gives sample statistics modindices(5.0): gives modication indices above 5 residual: gives residual matrix (distances between expected and observed variance-covariance matrix) standardized: gives standardized parameters Tech 1: gives parameters specication and starting values (useful only for debugging)
Mplus input: syntax(3)

parameter
syntax
example LV; [LV]; obs; [obs]; LV by obs1 obs2; LV1 with LV2; obs1 with obs2; LV1 on LV2 (Y on X); LV by obs1@1 obs2@1; LV1(1); LV2(1);
LV variance LV mean [] obs measurement error (variance) [] obs intercept loadings by correlations with regressions xed parameters parameters xed equal on @ (x)
From Eid, 1994

Well-being: likert scale (1 to 5) happy unhappy good not good well unwell content not content
N = 501 Mean age = 31.2 years (min. 17, max. 78) 292 females, 211 males
Mplus output
SAMPLE STATISTICS: means of observed variables and observed variance-covariance and correlation matrices TESTS OF MODEL FIT: 2 , CFI/TLI, loglikelihood, information criteria, RMSEA, SRMR MODEL RESULTS: parameters: estimates (unstandardized) standard error (S.E) ratio of estimates on S.E estimates (standardized by the parameter) estimates (fully standardized)
RESIDUAL OUTPUT: (observed estimated) MODIFICATION INDICES: for each additional parameter, decrease in 2 TECH OUTPUT
Diagram
Introduction Psychometrics SEM: Theory SEM: Examples LST: Theory Diagram Observed variables True score variable Latent variables Visually Representation Variance Uniqueness Meaningfulness Testability Identiability LST: Example MTMM: Theory MTMM: Examples MC Study: Theory MC Study: Example
Y21 Y11
Measures
Y12
Y22
Y13
Y23
Y14
Y24
Yi
Diagram
Y24 Y14 Y11
Measures
Y21
Y12
Y22
Y13
Y23
Yil
Diagram
Y24 Y11
Measures are inuenced by stable traits
1 T1 1 1 1
21
Y21
Y12 22 Y22
23
Y13
Y23 24 Y14
Yil = il + il T
Diagram
Y24 Y11
Measures are inuenced by stable traits
1 T1 1 1 1
21
Y21
Y12 22 Y22
1 IST2
23 1 1 1 24
Y13
Y23
Y14
Yil = il + il T + iil ISTi

Diagram
Y24 Y11 1 O1 1 T1 1 1 1 23 1 1 1 24 Y14 1 O4 1 Y23 1 22 Y22 1 21 Y21 1
Measures are inuenced by stable traits and momentary occasions
Y12
1 O2
1 IST2
Y13
1 O3
Yil = il + il T + iil ISTi + il Ok

Diagram
Y24 Y11 E11 1 O1 1 T1 1 1 1 23 1 1 1 24 Y14 1 O4 1 Y23 1 22 Y22 1 21 Y21 1
Measures are inuenced by stable traits and momentary occasions but there is still an error of measurement
Y12
1 O2
1 IST2
Y13
1 O3
Yil = il + il T + iil ISTi + il Ok + Eil

Observed variables
Introduction Psychometrics SEM: Theory SEM: Examples LST: Theory Diagram Observed variables True score variable Latent variables Visually Representation Variance Uniqueness Meaningfulness Testability Identiability LST: Example MTMM: Theory MTMM: Examples MC Study: Theory MC Study: Example Steyer, R. (1992) AND one of the other
The values of one random variable Yil are the scores on an item or scale i on an occasion l. The values of the mapping pU : U are the observational units u U (e.g., individuals). pU () = u, for each .
The values of the mappings pSitl : Sitl , l = {1, . . . , p} are the situations sitl Sitl : pSitl () = sitl , for each .
Observed variables (2)

Both the persons and the situations are supposed to be picked at random during the experiment. pU () and pSitl () are assumed to be independent. Assumption: Occasions cause random uctuations around the trait. This implies: The mean of the occasion-specic variables is zero. The correlations between trait and occasion-specic variables are zero.
True score variable

The latent true-scores Sil are dened as the expectation of the observed variables given the individuals and the situations: Sil := E(Yil | pU , pSit ). Sil characterizes a person-in-a-situation on an indicator i on an occasion l. The residual variables can be dened as the observed variables minus the conditional expectation presented above: Eil := Yil E(Yil | pU , pSit ). The proof that residuals are necessarily uncorrelated with the independent variable of the regression is given in Steyer, 1988.
Latent variables
The observed variables Yil can each be decomposed into: Yil = Sil + Eil = E(Yil | pU ) + [E(Yil | pU , pSit ) E(Yil | pU )] + Eil = Til + Oil + Eil , where Til := E(Yil | pU ) and Oil := [E(Yil | pU , pSit ) E(Yil | pU )]. Til characterizes the person itself across situations. The residual Oil represents eects of the situations and/or person-situation-interactions.
Visually
E11 T1 Y11 O
T1 IST T1
Y21
Y12
T1 IST T1
Y22
Y13
T1 IST T1
Y23
Y14
T1 IST
Y24
Representation
Assumption of a common linear factor for each type of latent variable (T , IST and O): Til = il + Til Ti , E(Til | T1l ) = ISTil ISTi , Oil = Oil Ol . The complete linear regression determining the observed variables Yil is then: Yil = il + Til Ti + ISTil ISTi + Oil Ol + Eil where ISTi is the indicator-specic deviation from the common trait T1 . ISTi = 0 when i = 1.
Decomposition of variance
V ar(Yil ) = 2 il V ar(Ti )+2 il V ar(ISTi )+2 il V ar(Ol )+V ar(Eil ) T IST O The decomposition contains no covariance because, for each observed variable, the latent variables are independent. Denition of several variance component coecients. The coecients are computed for each observed variables. One coecient concerns measurement error: U nrel(Yil ) = Conversely, the reliability is: rel(Yil ) = 1 U nrel(Yil ).
V ar(Eil ) . V ar(Yil )
Decomposition of variance: stable dispositions

Con(Yil ) = ComCon(Yil ) = M etSpe(Yil ) =
2 il V ar(Ti ) + 2 i V ar(ISTi ) IST T V ar(Yil ) 2 il V ar(Ti ) T , V ar(Yil ) 2 i V ar(ISTi ) IST . V ar(Yil )
The consistency coecient Con(Yil ) represents the proportion of stable disposition in the psychological score Yil . The common consistency coecient ComCon(Yil ) represents the proportion of stable disposition due to the common trait in the psychological score Yil . The method specicity coecient M etSpe(Yil ) represents the proportion of stable disposition due to the indicator-specic trait in the psychological score Yil .
Decomposition of variance: momentary inuences

OSpe(Yil ) =
2 il V ar(Ol ) O V ar(Yil )
The occasion specicity coecient OSpe(Yil ) represents the proportion of situational inuences in the psychological score Yil .
Uniqueness
If the latent variables of the model dene an LST model, any linear transformation of these latent variables will still yield an LST model. Ti = Ti + Ti Ti , Til , Ti
Ol = Ol Ol , il = il Ti Til Oil = Til /Ti , = Oil /Oil ,
where Ti , Ti , Oil , R and Ti , Oil > 0 For example: the observed variables measure skin temperature. The latent trait variable can be scaled on Celsius or Fahrenheit degrees and still be an LST model.
Meaningfulness
Ti (1 ) Ti (2 ) Ti (1 ) Ti (2 ) = , Ti (3 ) Ti (4 ) Ti (3 ) Ti (4 ) for [Ti (3 ) Ti (4 )] and [Ti (3 ) Ti (4 )] = 0. The ratio of the dierence of the value of two individuals u1 and u2 on the trait variable Ti on the dierence of the value of two other individuals u3 and u4 on this trait variable is equal to the same ratio for the same individuals on a transformed trait variable. Ol (1 ) Ol (1 ) , = Ol (2 ) Ol (2 ) for Ol (2 ) and Ol (2 ) = 0. The occasion-specic deviation value Ol (1 ) of a person u1 is ntimes (larger or smaller than) the value Ol (2 ) of a person u2 .
Meaningfulness(2)
2 2 il V ar(Ti ) = Til V ar(Ti ), T
Cor(Ti , Ti ) = Cor(Ti , Ti ),
2 2 il V ar(Ol ) = Oil V ar(Ol ), O
Although variances of latent variables by themselves are not meaningful, variances of latent variables multiplied by their squared loadings are. These products are used for consistency and specicity coecients. The correlations between latent trait variables are also meaningful and can therefore be interpreted.
Testability
Cov(Eil , Ti ) = 0, Cov(Eil , Ol ) = 0, Cov(Ti , Ol ) = 0, Cov(Ol , Ol ) = 0, Cov(E(il) , E(il) ) = 0.
Testability (2)
From the description of the null covariances, it follows that the covariance structure of the model is: = T T T + O O O +
T is the vector of loadings of the trait variables on the observed variables, O is the vector of loadings of the occasion-specic variables on the observed variables, T is the matrix of variances and covariances between trait variables O is the matrix of variances and covariances between occasion variables is the diagonal matrix of the measurement error variables
Identiability
Procedure: 1. Write all the equations of the expected variances and covariances. 2. Solve the system of v(v+1) equations with p unknown param2 eters. NO PROGRAM CAN SOLVE THESE SYSTEMS OF NON-LINEAR EQUATIONS. However, the identication conditions of the LST models have already been studied: at least three indicators of the same construct measured at three dierent occasions minimum of nine observed variables.
Questions: LST models

What assumptions are necessary for the LST model to be dened? For each assumptions, how can one test if they are fullled? What does the true score represent? And the error? Without the assumption of common linear factors, how many latent variables are there for an LST model with 3 indicators and 3 occasions of measurement and ISTs? What is the advantage of the additive decomposition of variance? Describe a model that does not have an additive decomposition of variance. Describe the variance component coecients. Write all variance-covariance equations of an LST model with 2 indicators and 2 occasions of measurement and IST.
Analysis procedure
Introduction Psychometrics SEM: Theory SEM: Examples LST: Theory LST: Example Analysis Model comparison M+ input: LST Results: sample Results: gof Results: variance Results: param. Exercises MTMM: Theory MTMM: Examples MC Study: Theory MC Study: Example
Begin by the less restricted model estimate all admissible parameters WARNING: This is not the saturated model since LST theory sets some parameters to zero. free model with one common trait and one indicatorspecic trait per indicator above one. Estimate: all loadings except one per latent variable all intercepts except one for the common trait all variances of the latent variables mean of the commmon trait variable error variables If more than 2 indicators, estimate correlations between IST variables.
Analysis procedure (2)

End with most restricted model most restricted model with one common trait. Estimate: all loadings xed to one all intercepts xed to zero variance of the trait variable variances of the occasion-specic variables xed equal mean of the commmon trait variable error variables xed equal
Model comparison
The best model should be as simple as possible but no simpler (freely inspired from Einsteins sentence) 2 dierence test between nested models using Maximum Likelihood: Compute 2 dierence and dierence of df If the test is not signicant, the more restricted model does not have a worse t than the larger model but is more parsimonious Choose the more restricted model If the test is signicant, the more restricted model has a worse t than the larger model Keep the larger model Note: for Maximum Likelihood Robust, the ML chi-square is equal to the MLR chi-square times the scaling correction factor
Mplus input: LST model

1 Freeest model pu free.inp pu fixload.inp 2 Fix the loading equal to one except for the loading of the common trait variables on the second indicator observed variables 3 Fix also the intercepts to zero expu fixload fixint.inp cept for the intercepts of the second indicator observed variables 4 Model with occasion-specic varipu fixload fixint =varO.inp ables variances xed equal 5 Model with all measurement erpu fixload fixint =varO =El.inp rors xed equal 6 Model with the rst measurement pu fixload fixint =varO =E1 =El.inp errors xed equal and all the other measurement errors xed equal
Mplus input: LST model (2)

Model 2 is measurement invariant for the common trait the common trait has the same meaning across time Model 3 is measurement invariant for the common trait and the common indicator-specic trait Model 4 is measurement invariant for the trait and occasionspecic variables Model 5 and 6 are perfectly measurement invariant
Results: sample statistics

P U11 P U21 P U12 P U22 P U13 P U23 P U14 P U24 Mean 15.02 14.60 15.09 14.85 15.01 14.67 15.50 15.09 P U11 12.23 0.82 0.22 0.22 0.23 0.25 0.26 0.23 P U21 10.51 13.46 0.23 0.29 0.22 0.29 0.22 0.24 P U12 2.80 3.04 13.32 0.89 0.28 0.30 0.34 0.32 P U22 2.94 3.95 12.22 14.14 0.29 0.35 0.33 0.35 P U13 2.97 2.99 3.74 4.02 13.64 0.89 0.33 0.37 P U23 3.18 3.85 4.02 4.78 12.03 13.41 0.36 0.43 P U14 3.11 2.76 4.27 4.27 4.25 4.51 11.89 0.88 P U24 2.83 3.15 4.15 4.63 4.76 5.51 10.64 12.37
Note: P Uil = mood for test-half i and occasion of measurement l; Variances are depicted on the main diagonal in bold type; Covariances appear in the lower triangle; Correlations are given in the upper triangle in italics.
Results: goodness-of-t statistics

Model ID 2 df p CF I RMSEA 2 df p CAIC BIC aBIC di M1: one trait 65.9 16 .00 0.98 .08 18590.3 18708.4 18619.5 M2: M1 plus indicator- 10.3 12 .59 1.00 .00 51.5 4 .00 18701.1 18669.1 18567.5 specic trait M3: M2 plus measurement 34.7 30 .25 1.00 .02 24.5 18 .14 18598.3 18584.3 18539.9 invariance with respect to the loadings, and occasionspecic variances M4: M3 plus measurement 46.7 36 .11 1.00 .02 12.4 6 .05 18567.5 18559.5 18534.1 invariance with respect to the mean structure M5: M4 plus autoregressive 42.8 32 .10 1.00 .03 4.1 4 .40 18591.3 18579.3 18541.3 structure
Note: CAIC = Consistent Akaike Information Criterion, BIC = Bayesian Information Criterion; aBIC = sample-size adjusted Bayesian Information Criterion; The 2 di- test refers to the 2 -dierence test comparing a model with a model in the preceding line. The appropriate approach for the MLR estimation method has been used. Please note that model M1 is nested within Model M2, and that model M4 is nested in Model M5. In the two other cases a model is nested in a model that is presented in the preceding line.
Results: variance components

P U11 P U21 P U12 P U22 P U13 P U23 P U14 P U24
Con(PUil ) ComCon(PUil ) SpeCon(PUil ) OSpe(PUil ) Rel(PUil ) .27 .27 .00 .58 .85 .31 .27 .04 .55 .86 .28 .28 .00 .62 .90 .32 .28 .04 .59 .91 .28 .28 .00 .62 .90 .32 .28 .04 .59 .91 .28 .28 .00 .62 .90 .32 .28 .04 .59 .91
Note: P Uil = Test halves; i = test half; l = occasion of measurement; Con(P Uil ) = consistency coecient; ComCon(P Uil ) = common consistency coecient; SpeCon(P Uil ) = occasion-specicity consistency coecient; Rel(P Uil ) = reliability coecient (Due to rounding errors, the sums of the consistency and occasion specicity coecients sometimes do not always sum to the reliability coecients).
Results: parameters
Y24 1 1 1.02 (0.03) Y11
2.00 (0.17)
Means
Trait Trait Variances Loadings
Error Variances
2.00 (0.17)
Occasionspecific Variances
Means
7.76 (0.42)
1
Y21
1.23 (0.07)
0.64 (0.13) 15.15 (0.11) 3.55 (0.44)
1 1 1
1.02
Y12
1.23 (0.07)
7.76 (0.42)
1
Y22
1.23 (0.07)
1 1
1.02
Y13
1.23 (0.07)
7.76 (0.42)
1
0.56 (0.10)
1
1.02
Y23
1.23 (0.07)
Y14
1.23 (0.07)
7.76 (0.42)
1
Standard errors in parentheses, indicated only once for trait loadings. Intercepts of the model: 0.70(.48) for the i = 2 indicator variables.
Exercises
Data from Cole (1995) on depression and anxiety in children. Depression measured by CDI for children self-report (26 items). The scale was split in two test-halves of 13 items. Depression measured by TRID for teacher-report (26 items). The scale was split in two test-halves of 13 items. Anxiety measured by RCMAS for children self-report Anxiety measured by TRIA for teacher-report TRID and TRIA are teachers version of the childrens scales. n = 375 Subjects: children in grade 4 and then 5. Subjects were measured once a semester on 4 semesters. variables are presented in this form: ds th1 1 where the rst letter is for depression or anxiety, the second letter is for self or teacher rating, the second part is for Test-Half 1 or 2 and the third part is for wave (1, 2, 3 or 4).
Exercises
Analyze with LST models the following data set (cole-4o-dast-miss.dat). You can choose either depression or anxiety as the trait (but not both). Please choose self-report and not teacher report. A draft of the input can be found on dokeos: adst empty miss.inp. Try to nd the most parsimonious and yet precise and interpretable model. Note: The commands CLUSTER IS class; and TYPE IS complex; are necessary because the data is multilevel. This subject will not be discussed in this course. Just let the commands and proceed with your LST models. Question: Interpret thoroughly your ndings Question: Explain the path you took to restrict your model and why you took this path.
MTMM models: generalities

Introduction Psychometrics SEM: Theory SEM: Examples LST: Theory LST: Example MTMM: Theory MTMM models Choice of model CT CTCU CTC(M1) MTMM: Examples MC Study: Theory MC Study: Example
There exist more than 20 dierent MTMM models. Methods can be any kind of repeated measurement for the same construct: raters occasions of measurement in fact, any kind of sources of inuence presented on page 22 except trait. MTMM models are useful to determine convergent and discriminant validity.
Eid, Lischetzke and Nussbeck (2006) AND one other
Choosing an MTMM model

Distinctions to choose the model most appropriate for a specic data set: Single indicator vs. multiple indicators MTMM models. If possible, always choose multiple indicators models because they allow a separation of measurement error from methodspecic variables. Allows correlated methods or not. MTMM correlation (Correlated Trait CT) model Model deviations from the trait or model specic method factors.
Choosing an MTMM model (2)

One method can be chosen as a reference or not. If methods are interchangeable (for example 3 students rate their professor), no method can be chosen as reference (in this case, raters are considered as methods). The mean is then taken as reference. Correlated Trait Correlated Uniqueness (CTCU) model If one method can be chosen as a reference (for example self-rating and mother- and father-ratings), the self-rating can be taken as reference (again, raters are considered as methods). Correlated Trait Correlated Method 1 (CTC(M1)) model corresponds to the two types of contrasts in ANOVA.
Correlated trait model

allows: estimation of the latent correlations between the traitmethod units the construction of a latent MTMM correlation matrix (error-free variant of the MTMM matrix) does not allow: separation of trait and method eects estimation of general method eects estimation of general trait eects
Correlated trait model: diagram

Y113 Y213 Y123 Y223 1 M213 M.13 Y211 Y121 Y221 Y112 Y212 Y122 Y222 Y111 E111 1 M211 M.11
1 M221 222 M.21
1 M212 213 M.12
1 M222 223
1 M.22 M 222
1 M223 M.23
Correlated Trait Correlated Uniqueness model

allows: estimation of common trait and discriminant validity estimation of the generalizability of method eects within traits (by correlations between method factors belonging to the same trait) inclusion of covariates on the method level does not allow: estimation of a common method factor estimation of the generalizability of method eects across traits (correlations of method factors belonging to dierent traits not allowed) quantication of trait, method and error inuences
Correlated Trait Correlated Uniqueness model (2)

can be extended to: Correlated Trait Uncorrelated Method (CTUM): if the method factors belonging to the same method are identical Correlated Trait Correlated Method (CTCM): if CTUM and the method factors can correlate. These models are very restrictive because they imply a perfect unidimensionality of the method inuences belonging to the same method.
Correlated Trait Correlated Uniqueness model: diagram

Introduction Psychometrics SEM: Theory
Y211 Y111 E111 1 M211 M.11
SEM: Examples LST: Theory LST: Example MTMM: Theory MTMM models Choice of model CT CTCU CTC(M1) MTMM: Examples MC Study: Theory MC Study: Example
T22. T11.
1 Y121 T112 T113 1 T21. T212 T213 1 T12. T122 T123 1 T222 T223 Y222 Y113 Y213 Y123 Y223 Y112 Y212 Y122 1 M212 M.12 Y221 1 M222 221 M.21
1 M222 M.22
1 M213 M.13
1 M223 M.23
Correlated trait Correlated Method 1 model

allows: estimation of the generalizability of method eects within traits (by correlations between method factors belonging to the same trait) estimation of the generalizability of method eects across traits (by correlations between method factors belonging to dierent traits) quantication of trait, method and error inuences estimation of heteromethod coecients of discriminant validity (by correlations between method factors of a trait and trait factor of another trait) does not allow: estimation of method eect for the standard method This model can be used only if one method can be chosen as a reference.
Correlated trait Correlated Method 1 model: diagram

T22. 1 Y113 Y213 Y123 Y223 1 M.13
a b a
E111 Y111 Y211 1 T11. Y121 Y221 1 T21. Y112 Y212 1 T12. Y122 Y222 1 M.12
b
1 M.22
1 M.23
From Kocum, 2006

Introduction Psychometrics SEM: Theory SEM: Examples LST: Theory LST: Example MTMM: Theory MTMM: Examples Data Analysis Results Exercises: Cole MC Study: Theory MC Study: Example
Traits: Work competence (5 items) Activity/potency (10 items) Democratic Approach (8 items) measured for: manager female manager covariate: Neosexism (11 items) data with all items: Kocum2006.dat
Analysis
1. Create the test-halves Do a factorial analysis with one factor on the variables of the reference method assign the item with highest loading to the rst test-half assign the item with second-highest loading to the second test-half ... compute the mean of each test-half 2. Decide which model(s) is(are) the most appropriate to answer your research questions. Interpretation: For each model, the same parameter (for example, correlation between traits) may have a completely dierent interpretation (either convergent or discriminant validity in CT but only discriminant validity in CTC(M1)).
Analysis (2)
CTC(M1) model freest model: kocum free dc.inp
based on the results, restrict the model and compare model t with 2 -di test interpret the most parsimonious yet precise model CTC(M1) model with neosexism as method covariate freest model in Mplus kocum free cov dc.inp
interpret correlations between method factors and neosexism.
Results
What must be presented: sample statistics: vector of means, correlation and covariances matrices goodness of t indices variance components parameter estimates relations with covariates
Exercises: Cole
CTUM model of Coles data (rst wave only) input the model in Mplus present the results indicate which results could not have been obtained by the CT model Are the results concordant with the results of the other models? CTCM model input the model in Mplus present the results indicate which results could not have been obtained by the CT and CTUM model Are the results concordant with the results of the other models?
Why do Monte Carlo studies of applications?

Introduction Psychometrics SEM: Theory SEM: Examples LST: Theory LST: Example MTMM: Theory MTMM: Examples MC Study: Theory Why Procedure M+ input: MC M+ output: MC MC Study: Example
MOST IMPORTANTLY: Can one trust the parameters estimates? for LST: How many occasions of measurements are needed to obtain valid results? What is the minimum sample size required to apply the model? Compare these minimum requirements with your applications sample size and, for LST, number of occasions.
Muthn, L.K., Muthn, B.O. (2002) e e
Procedure for the Monte Carlo studies

1. Specify a population with parameters equal to the empirical application. 2. Generate R replications (for ex. 500) of the population values. 3. To each replication, add a random noise so that each data will be slightly dierent. Then estimate model parameters with the same model as the empirical model. May specify starting values equal to those of the empirical application. 4. May change number of observations or, for LST, number of occasions
Mplus input: Monte Carlo study

MONTECARLO: NAMES ARE: names of the observed variables NOBSERVATIONS =: number of observations NREPS =: number of replications (usually 500) SAVE = xxx.sav: complete result for each replication (data values) can be saved in an spss le RESULTS = xxx.sav: summary of results can be saved in an spss le
MODEL MONTECARLO: ALL parameters must be specied with a true value. These values are the one estimated in the empirical application. (example: T1*5.34; means the true variance of T1 is 5.34. * or @ can be used.) ANALYSIS: same as in the empirical application. MODEL: same as in the empirical application. OUTPUT: tech8 is useful to see computation progress.
Mplus output: Monte Carlo study

SUMMARY OF ANALYSIS, NUMBER OF REPLICATIONS: indicates requested and completed replications. If model presents estimation problems, many replications will not be completed. SAMPLE STATISTICS FOR THE FIRST REPLICATION: all sample stat can be obtained if the SAVE option was used in the input. TESTS OF MODEL FIT: indicates, for each goodness-of-t criteria, standard deviation and number of succesful computations.
Mplus output: Monte Carlo study

MODEL RESULTS: for each parameter, indicates: POPULATION: population value inputed. ESTIMATES AVERAGE: average of the parameters estimate across replications. STD. DEV.: standard deviation for each parameters estimate. S.E. AVERAGE: average of the standard error across replications. M.S.E.: Mean Squared Error: (population estimates average)2 + Std.Dev.2 95% COVER: percentage of replications in which the population value was in the 95% CI around the parameters estimate. % SIG COEFF: percentage of replications in which the 95% CI around the parameters estimate did not include zero (= 0 for xed parameters).
Exercises: Monte Carlo study

input a Monte Carlo study for the best LST model (Eid, 1994 data) are the goodness of t criteria and parameters estimate appropriate? Justify. input a Monte Carlo study for the best MTMM model (Kocum, 2006 data) are the goodness of t criteria and parameters estimate appropriate? Justify.
Monte Carlo study: MSE

Introduction Psychometrics SEM: Theory SEM: Examples LST: Theory LST: Example MTMM: Theory
mean MSE 80 140 occasions 120 2 3 4 5 6
MTMM: Examples MC Study: Theory MC Study: Example MSE coverage
20
40
60
100
125
250
500
1000 observations
2000
Monte Carlo study: coverage

Introduction Psychometrics SEM: Theory SEM: Examples
0.98 1.00 occasions
LST: Theory LST: Example MTMM: Theory MTMM: Examples MC Study: Theory MC Study: Example MSE coverage
mean coverage
2 3 4 5 6
0.90
0.92
0.94
0.96
125
250
500
1000 observations
2000

Psycho Metrics 08

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Psycho Metrics 08

Uploaded by

Copyright:

Available Formats

Questionnaires and measures validation: Psychometric models: Latent State-Trait models and Multitrait-Multimethod models

Delphine Courvoisier, Olivier Renaud Section of Psychology University of Geneva

Delphine Courvoisier & Olivier Renaud

Psychometric models summer 07 1

Materials for course and practicals

Delphine Courvoisier & Olivier Renaud

Psychometric models summer 07 3

Delphine Courvoisier & Olivier Renaud

Psychometric models summer 07 4

Delphine Courvoisier & Olivier Renaud

Psychometric models summer 07 5

Bibliography: Structural Equation Models

Bibliography: Latent State Trait theory and models

Delphine Courvoisier & Olivier Renaud

Psychometric models summer 07 7

Bibliography: Multitrait Multimethod theory and models

Bibliography: Monte Carlo studies

Delphine Courvoisier & Olivier Renaud

Psychometric models summer 07 9

Delphine Courvoisier & Olivier Renaud

Psychometric models summer 07 13

Graphical representation of reliability and validity

Validation of questionnaire: Why?

Delphine Courvoisier & Olivier Renaud

Validation of questionnaire: How?

Classical test theory

Delphine Courvoisier & Olivier Renaud

Graphical representation of measurement error

Graphical representation of measurement error

may vary as a function of:

may vary as a function of:

A psychological score Yjk may vary as a function of:

Stable construct j Method of measurement k

A psychological score Yjkl may vary as a function of:

Stable construct j Method of measurement k Momentary situations l

A psychological score Yjklc may vary as a function of:

Stable construct j Method of measurement k Momentary situations l Structural dierences c

A psychological score Y jklc + may vary as a function of:

Theoretically, each set of sources of inuence can be modeled.

Conrmatory factor analysis (CFA) and structural equation models (SEM)

Delphine Courvoisier & Olivier Renaud

Psychometric models summer 07 25

Basic principles of SEM

Null hypothesis in SEM

Delphine Courvoisier & Olivier Renaud

Psychometric models summer 07 29

Advised set of of indices

RMSEA and condence interval SRMR NNFI CFI

Delphine Courvoisier & Olivier Renaud

Psychometric models summer 07 30

Steps to test SEM

Submodels of complex SEM

Delphine Courvoisier & Olivier Renaud

Psychometric models summer 07 32

Factor identication in SEM (scaling)

Delphine Courvoisier & Olivier Renaud

Psychometric models summer 07 33

Informations often missing in articles using SEM

Delphine Courvoisier & Olivier Renaud

Psychometric models summer 07 34

Delphine Courvoisier & Olivier Renaud

Psychometric models summer 07 35

Hershberger, 2003 (suite)