Instrumentation and

Data Collection
Muhammad Luqman Qadir
M.Phil. (Molecular Biology and
Forensic Science)
MOST COMMONLY USED TOOLS
• Questionnaires
• Interviews
• Observations
• Tests ( achievement, aptitude,
personality)
• Scales
• Document Analysis
TESTS
Tests are measurement tools which are employed to measure the performance
of an individual in some specific area of interest.
TYPES OF TESTS
There are Two main types:

• Norm referenced /Standardized Tests

• Criterion Referenced Tests

NORM REFERENCED TESTS
• These are used relatively to measure the performance of some Individual
• Norm is basically the standard to compare the performance with.
TYPES OF NORM REFERENCED TESTS
The most commonly exploited types are;

1. Achievement Test

2. Aptitude Test

3. Intelligence Test

4. Personality Test

5. Vocational /Career Test

6. Sensory Motor Test

1. ACHIEVEMENT TEST

• “An instrument that measures an individual’s current proficiency

in given areas of knowledge or skill.”
• The performance is interpreted in the form of grades to compare
• with the performance and learning of others.
2. APTITUDE TEST
• A measure of potential used to predict how well an individual is likely to
perform in a future situation.”
• Such test measures the potential of some individual in learning about what
they can learn not about what they have already learned.
3. INTELLIGENCE TEST
Such a test is used to measure the mental abilities
i.e.

• to analyze the situations,

• to create meaning,

• to organize the ideas and

• to measure the abstract intelligence of an

individual
4. PERSONALITY TEST
This type of test is resorted to

• measuring the types or characteristics of the personality and

• measuring the human behaviors, their skills and their needs

etc.
5. VOCATIONAL / CAREER TEST
• This sort of test is utilized to measure the ability and suitability
of
some person in taking up some profession and career.
6. SENSORY MOTOR
TEST
This test is used:

• to measure an individual’s sensory capabilities and motor

abilities

• to measure the sensory coordination

CRITERION REFERENCED TEST

This type of tests interprets an individual’s performance on the basis of an

absolute criteria.

• Criterion is basically some specific subject or skill

• Main purpose of this test is to measure some specific objectives which are
pre-defined operationally and behaviorally.
Content validity of these tests is required.

▶ E.g. There is no restriction about the number of individual s

falling in pass marks criterion.

▶ The cut off or the passing marks criteria is set by either the
subject teacher or by the researcher.
These two testing types have different construction methods,
underlying goals, and methods for interpreting scores.

NORM-REFERENCED TESTS CRITERION-REFERENCED TESTS

• Norm-referenced tests make • criterion-referenced

tests
comparisons between measure a test
taker's
individuals. performance compared to
a specific set of standards
or criteria
SCALES
Here scales mean “Measurement Scales”

▶ These are used to collect the different forms of data and

quantify the response of the respondents to measure the
variables.

▶ Type of data determines the type of measurement scale

i.e.
nominal, ordinal, interval and ratio.
TYPES OF
SCALES
1. Nominal
scales

2. Ordinal scales

3. Interval scales

4. Ratio scales

5. Likert scales

6. Rating scales
NOMINAL SCALES
▶ The term means “to name”

▶ The categories differ in quality rather than quantity

▶ Each value belongs to only its own category but can be more or less than the
other

▶ Nominal scales are based on categories which must be mutually

exclusive, exhaustive and uni- dimensional

▶ Most elaborating scale

▶ There are no chance of addition, multiplication, subtraction or other
mathematical operations with the categories.

It provides :

▶ The ratio or the frequency and the definite information

▶ Can be used to measure Gender or preferences of respondents

NOMINAL SCALE EXAMPLE
ORDINAL SCALE
▶ The term means “to order”

▶ In ordinal scale the idea is organized in some order (high-low or low-

high)

▶ This scale does not specify how much different the categories are from
each other

▶ Categories can be presented in the form of a continuum

▶ Categories are given in a logical or a rank order

▶ Height, weight, income and the ranks can be measured

ORDINAL SCALE EXAMPLE
Your income is:
o Below 50,000
o 50,100 to
60,000
o 60,100 to
70,000
INTERVAL SCALE
▶ It is used to tell the order of the objects

▶ It shows how much distant the categories are from each other

▶ All types of mathematical operations can be performed on this data

▶ Rating scales are ed as the interval scales also

▶ The difference between the ages of people can be identified with

the
help of this scale
INTERVAL SCALE EXAMPLE
Kindly mention the number of employees at your firm from year 2005
to 2010:
Year Number of
• 2005 Employees
• 5
• 2006 • 7
• 10
• 2007 • 22
• 2008 • 25
• 2009 • 27
• 50
• 2010
 RATIO SCALE
▶ It is a scale with true value “0”

▶ Used to get quantitative data

▶ Salary quantities purchased and market share are all

expressed
on a ratio scale

▶ most sophisticated scale of all

LIKERT SCALE
▶ A uni-dimensional scale usually with five points

▶ All the points are of equal interval between each other

▶ Records the agreements or disagreement of respondent about

some particular variable

▶ Options are pre-coded

 RATING SCALES

▶ It is a scale used in observational studies

▶ It is used to record the behaviour or activity of the

respondents

▶ Descriptive statements are used with infinite points on a line

▶ Points or line to record the behaviour is termed as RATER.

▶ It can be subjected to human error

Rating scale example
OBJECTIVITY
• The degree to which a test’s results are obtained the same by
scoring different scorers without influences of their biases or beliefs
on scoring is known as objectivity.

• Most standardized tests of aptitude and achievement tests are high

in objectivity.
▶ In essay-type tests requiring judgmental scoring, different persons get different results
or even the same person can get different results at different times (Linn & Gronlund,
2000).

▶ Objective-type tests such as true/false, multiple-choice, and so

on are developed to overcome the lack of objectivity in tests.

▶ In essay-type tests, objectivity may be increased by careful phrasing of questions and by

a standard set of rules for scoring (Swain et al, 2000).
STANDARDIZATION
▶ Standardization is the process of creating standards to guide the creation of a good or
service based on the consensus of all the relevant.

▶ A standardized test is any form of test that requires all test takers to answer the same
questions, or a selection of questions from common bank of questions, in the same way,
and that is scored in a “standard” or consistent manner, which makes it possible to
compare the relative performance of individual students or groups of students.
RELIABILIT
Y
• Reliability refers to the extent
that the instrument yields the
same results over multiple
trials.
VALIDITY
• Validity refers to the extent
that the instrument measures
what it was designed to
measure.
 VALIDI
TY
• The accuracy with which a test measures whatever it is supposed to measure.

• An evaluation procedure is valid to the extent that it provides an assessment of

the degree to which pupils have achieved specific objectives , content matter and
learning experiences
• Validity is an important characteristic of any test. This refers to what the test really
measures. A test is valid, if it measures what we really wish to measure.
ASSESSMENT OF VALIDITY
In research, there are three ways to approach validity
and they include:
• content validity,
• construct validity,
• criterion-related validity.
Content Validity

• Content validity measures the extent to which the items that comprise
the scale accurately represent or measure the information that is being
assessed.
• Are the questions that are asked representative of the possible
questions that could be asked?
Construct validity
• Construct validity measures what the calculated scores mean and if
they can be generalized. Construct validity uses statistical analyses,
such as correlations, to verify the relevance of the questions.
• Questions from an existing, similar instrument, that has been
found reliable, can be correlated with questions from the
instrument under examination to determine if construct validity is
present. If the scores are highly correlated it is called convergent
validity. If convergent validity exists, construct validity is supported.
Criterion-related validity
• Criterion-related validity has to do with how well the scores from the
instrument predict a known outcome they are expected to predict.
Statistical analyses, such as correlations, are used to determine if
criterion-related validity exists.
• Scores from the instrument in question should be correlated with an
item they are known to predict. If a correlation of > .60 exists,
criterion related validity exists as well.
RELIABILITY
▶ Reliability is a characteristic of any test refers to the accuracy and consistency of
information obtained in a study.
▶ A well-developed scientific tool should give accurate results both at present as well
as over the time.
▶ A test good reliability means that the test taker will obtain the same test score over
repeated testing as long as no other extraneous factors have affected the score.
▶ A good instrument will produce consistent scores. An instrument’s
reliability is estimated using a correlation coefficient .
ASSESSMENT OF RELIABILITY
Reliability can be assessed with the:
1. Test-retest Method
2. Alternative Form Method
3. Internal Consistency Method
4. The Split-halves Method
5. Inter-rater Reliability
TEST-RETEST METHOD
• Test-retest is a method that administers the same
instrument to the same sample at two different points in
time, perhaps one year intervals.
• If the scores at both time periods are highly correlated,
> .60, they can be considered reliable
ALTERNATIVE FORM
METHOD
The alternative form method requires two different
instruments consisting of similar content. The same sample
must take both instruments and the scores from both
instruments must be correlated. If the correlations are high,
the instrument is considered reliable.
INTERNAL CONSISTENCY
METHOD
Internal consistency uses one instrument administered only
once. The coefficient alpha (or Cronbach’s alpha) is used to
assess the internal consistency of the item. If the alpha value
is .70 or higher, the instrument is considered reliable

Video link (Cronbach’s Alpha): https://youtu.be/W9uPvAmtTOk?si=mY3mbGI3iWKVGdH6

Cronbach’s
Alpha
THE SPLIT-HALVES METHOD
The split-halves method also requires one test administered
once. The number of items in the scale are divided into
halves and a correlation is taken to estimate the reliability of
each half of the test. To estimate the reliability of the entire
survey, the Spearman-Brown correction must be applied.
INTER-RATER RELIABILITY
Inter-rater reliability involves comparing the observations of two or more
individuals and assessing the agreement of the observations. Kappa
values can be calculated in this instance.
DEVELOPMENT OF TESTS/SCALES
DEVELOPMENT OF TESTS/SCALES

The process of creating a new instrument for measuring an unobserved

or latent construct, such as depression, sociability, or fourth-grade
mathematics ability.
1. Make a full and clear statement of the testing ‘problem’.

2. Write complete specifications for the test.

3. Write and moderate items.

4. Trial the items informally on native speakers and reject or

modify problematic ones as necessary.

5. Trial the test on a group of non-native speakers similar to

those for whom the test is intended.
6. Analyse the results of trial and make any
the necessary changes.
7. Calibrate scales.
8. Validate.
9. Write handbooks for test takers, test users and staff.
10. Train any necessary staff (interviewers, raters, etc.).
TESTS/SCALE DEVELOPMENT PROCESS
QUESTIONS TO BE ANSWERED DURING PLANNING OF THE
TESTS/SCALE DEVELOPMENT
ITEM SPECIFICATIONS TO CONSIDER ESPECIALLY IN LIKERT AND
LIKERT-TYPE SCALES
Any Question?