Educational Research: Instruments (³caveat emptor´) (³caveat emptor´


EDU 8603 Educational Research Richard M. Jacobs, OSA, Ph.D.


researchers use to collect data for research studies (alternatively called ³tests´)

The types of instruments«
1. Cognitive Instruments 2. Affective Instruments 3. Projective Instruments

Cognitive instruments.  Measure an individual¶s attainment in academic areas typically used to diagnose strengths and weaknesses .1...

 achievement tests «provide information about how well the test takers have learned what they have been taught in school «achievement is determined by comparing it to the norm the norm. performance of a national group of similar students who have taken the same test .Types of cognitive instruments...

and a quantitative score . a verbal score. aptitude tests «measure the intellect and abilities not normally taught and often are used to predict future performance «typically provide an overall score.

and a variety of other activities.2. and attitudes toward self. Affective instruments..  Measure characteristics of individuals along a number of dimensions and to assess feelings. and situations . others.. institutions. values.

. and situations «frequently use Likert.  attitude scales «self-reports of an individual¶s beliefs. others. or Guttman scales .. perceptions. Thurstone . institutions.Types of affective instruments. and a variety of activities. semantic differential. or feelings about self.

economic. political. and religious values . social. aesthetic. values tests «measure the relative strength of an individual¶s valuing of theoretical.

 personality inventories «an individual¶s self-report measuring how behaviors characteristic of defined personality traits describe that individual .

.. Projective instruments.  Measure a respondent¶s feelings or thoughts to an ambiguous stimulus .3.

Primary type of projective test... 

tests «participants react to a stimulus such as a picture, inkblot or word onto which they project a description

Selecting an instrument...
1. determine precisely the type of instrument needed 2. identify and locate appropriate instruments 3. compare and analyze instruments 4. select best instrument

Instrument sources«
Burros¶ Mental Measurements Yearbook Tests in Print PRO-ED Publications Test Critiques Compendium ETS Test Collection Database ERIC/AE Test Review Locator ERIC/Burros Test Publisher Directory

test takers¶ lack of familiarity with instrument 5. 1.. the greatest ease of administration. the highest reliability 3. scoring. avoids potentially controversial matters . and interpretation 4. the highest validity 2..Rules governing the selection instruments.

1.. make arrangements in advance 2. ensure ideal testing environment 3. be prepared for all probable contingencies .Administering the instrument..

Reliability the degree to which the Reliability: instrument consistently measures what it purports to measure . 1..Two issues in using instruments. Validity the degree to which the Validity: instrument measures what it purports to measure 2..

Content validity 2.. 1. Criterion-related validity 3.Types of validity. Construct validity ..

Content validity the degree to which validity: an instrument measures an intended content area .1.

forms of content validity« validity «sampling validity does the instrument sampling validity: reflect the total content area? item validity: «item validity are the items included on the instrument relevant to the measurement of the intended content area? .

2. Criterion-related validity: an Criterionvalidity individual takes two forms of an instrument which are then correlated to discriminate between those individuals who possess a certain characteristic from those who do not .

forms of criterion-related validity« criterionvalidity «concurrent validity: the degree to which scores on one test correlate to scores on another test when both tests are administered in the same time frame «predictive validity the degree to which a predictive validity: test can predict how well individual will do in a future situation .

3. Construct validity a series of studies validity: validate that the instrument really measures what it purports to measure .

Equivalence 3...Types of reliability. 1. Internal consistency . Stability 2.

Stability (³test-retest´): the degree to which two scores on the same instrument are consistent over time .1.

Equivalence (³equivalent forms´): the degree to which identical instruments (except for the actual items included) yield identical scores .2.

KuderRichardson and Cronback¶s Alpha reliabilities.3. Internal consistency (³split-half´ reliability with Spearman-Brown correction formula . scorer/rater reliability): the degree to which one instrument yields consistent results .

Terms associated with instruments...

Data« Data «the pieces of information researchers collect through instruments to examine a topic or hypothesis

Constructs« Constructs «abstractions of behavioral factors that cannot be observed directly and which researchers invent to explain behavior

Variable« Variable «a construct that can take on two or more values or scores .

Raw scores scores« «the number of items an individual scored on an instrument .

Measurement scales scales« «the representation of variables so that they can be quantified .

ratio variables . interval variables 4. ordinal variables 3. Qualitative (categorical) 1...Measurement scales. nominal variables Quantitative (continuous) 2.

1. nominal (³categorical´): classifies persons or objects into two or more categories .

ordinal (³order´): classifies persons or objects and ranks them in terms of the degree to which those persons or objects possess a characteristic of interest .2.

and classifies interval: persons or objects according to equal differences with no true zero point .3. interval ranks. orders.

orders. ratio ranks.4. classifies persons ratio: or objects according to equal differences with a true zero point .

Norm reference reference« «provides an indication about how one individual performed on an instrument compared to the other students performing on the same instrument .

Criterion reference reference« «involves a comparison against predetermined levels of performance .

Self reference reference« «involves measuring how an individual¶s performance changes over time .

Operationalize« Operationalize «the process of defining behavioral processes that can be observed .

Standard error of measurement measurement« «an estimate of how often a researcher can expect errors of a given size on an instrument .

MiniMini-Quiz«  True or false« «a large standard error of measurement indicates a high degree of reliability false .

 True or false« «a large standard error of measurement indicates low reliability true .

 True or false« «most affective tests are projective false .

 True or false« «the primary source of test information for educational researchers is the Burros Mental Measurements Yearbook true .

 True or false« «research hypotheses are usually stated in terms of variables true .

a Guttman scale attempts to determine whether an attitude is unidimensional true . True or false« «similar to a Thurstone scale.

 True or false« «validity requires the collection of evidence to support the desired interpretation true .

 True or false« «researchers should first consider developing an instrument rather than utilizing a published instrument false .

 True or false« «a researcher¶s goal is to achieve perfect predictive validity false .

 True or false« «predictive validity is extremely important for instruments that are used to classify or select individuals true .

00 than 0.00 true . True or false« «a high validity coefficient is closer to 1.

 True or false« «norm reference and criterion reference are synonymous terms false .

 True or false« «³criterion related´ refers to correlating one instrument with a second instrument. the second instrument is the criterion against with the validity of the second instrument is judged false .

 True or false« «a valid test is always reliable but a reliable test is not always valid true .

like validity.. i. groups with different characteristics will produce different reliabilities true . True or false« «it is difficult to state appropriate reliability coefficients because reliability.e. is dependent upon the group being tested.

 True or false« «content validity is not compromised if the instrument covers topics not taught false .

 Fill in the blank« «the tendency of an individual to respond continually in a particular way response set .

 Fill in the blank« «a study which consists of two quantitative variables correlational .

 Fill in the blank« «a study which consists of one categorical and one quantitative variable experimental or causal-comparative .

 Fill in the blank« «a study which consists of two or more categorical variables correlational or descriptive .

 Fill in the blank« «data collection methods which emphasize student processes or products performance .

true-false. Fill in the blank« «data collection methods including multiple-choice. and matching selection .

provide a short answer. Fill in the blank« «data collection methods in which students fill in the blank. or write an essay supply .

and interpreted in the same way no matter where or when it is administered standardized . scored. Fill in the blank« «an instrument administered.

 Fill in the blank« «the term that includes the general process of collecting. whether formal or informal assessment . and interpreting information. synthesizing.

systematic. usually paper-and-pencil procedure for gathering information about peoples¶ cognitive and affective characteristics test . Fill in the blank« «a formal.

 Fill in the blank« «the degree to which individuals seek out or participate in particular activities. objects. and ideas interests .

´ the characteristics representing an individual¶s typical behaviors and describes what individual do in their natural life circumstances personality . Fill in the blank« «also called ³temperament.

the tendency to accept or reject groups. Fill in the blank« «things individuals feel favorable or unfavorable about. or objects attitudes . ideas.

or objects values . Fill in the blank« «deeply held beliefs about ideas. persons.

 Fill in the blank« «requires administering the predictor instruments to a different sample from the same population and developing a new equation cross-validation .

 Which type of test« «Minnesota Multiphasic Personality Inventory personality inventory .

 Which type of test« «Stanford-Binet achievement test .

 Which type of test« «Strong Campbell interest inventory .

 Which type of test« «SRA Survey of Basic Skills achievement test .

 Which type of test« «Weschler Intelligence Scales aptitude test .

 Which type of test« «Gates-McGinitie Reading Test achievement test .

 Which type of test« «Otis-Lennon School Ability Test aptitude test .

 Which type of test« «Kuder Occupational interest inventory .

 Which type of test« «Rorschach Inkblot Test projective .

 Which type of test« «Meyers-Briggs Type Indicator personality inventory .

 Which type of test« «Iowa Test of Basic Skills achievement test .

 Which type of test« «Thematic Apperception Test projective .

 Which type of validity« «compares the content of the test to the domain being measured content .

 Which type of validity« «Graduate Record Examination predictive .

either at the same or different time criterion-related . Which type of validity« «correlates scores from one instrument to scores on a criterion measure.

and content-related evidence to determine that the presumed construct is what is being measured construct . divergent. Which type of validity« «amasses convergent.

 Which type of reliability« «scores on one instrument are consistent over time stability (test-retest) .

 Which type of reliability« «the extent to which independent scorers or a single scorer over time agree on the scoring of an open-ended instrument scorer/rater .

 Which type of reliability« «scores correlate between similar version of an instrument given at different times equivalence and stability .

 Which type of reliability« «scores correlate between two versions of a test that are intended to be equivalent equivalence (alternate forms) .

 Which type of reliability« «the extent to which items included on an instrument are similar to one another in content internal consistency .

 Which type of response scale« «an individual gives a quantitative rating to a topic where each position on the continuum has an associated score value semantic differential .

 Which type of response scale« «value points are assigned to a participant¶s responses to a series of statements Likert .

 Which type of response scale« «participants select from a list of statements that represent differing points of view from those which participations agree Thurstone .

. instruments «which describes the procedures researchers use to select individuals to participate in a study ..This module has focused on.

..the tools researchers use to gather data for a study . qualitative research ..The next module will focus on..

Sign up to vote on this title
UsefulNot useful