DATA COLLECTION, MEASUREMENT, AND DATA  Easy to administer

QUALITY IN QUANTITATIVE AND QUALITATIVE  More efficient time use

RESEARCH
By: Mr. Mark Angelo T. Cristino, MAN, RN
I. DATA COLLECTION METHODS
 Without appropriate data collection methods,
the validity of research conclusions is easily
challenged.
a. Using new data
 Collect own data for the study.
b. Using existing data
 Historical research – use records and other
documents from the past
 Secondary analysis – use of data gathered
in a previous study.
Interviews vs. Questionnaires
Key Dimensions of Data Collection Methods;
1. Structure – the data collection should be very
structured and consistent.
2. Quantifiability – able to be analyzed statistically.
3. Obtrusiveness – degree to which people are
aware that they are being studied.
4. Objectivity – try to be as objective as possible.
DATA COLLECTION IN QUANTITATIVE RESEARCH
Types of Data Collection;
a. Self-reports
 Interviews
 Questionnaires
 Scales (Composite Scales – [Social-
Psychological])
 Vignettes
 Projective techniques
 Q-sorts
ii.
i. Interviews and Questionnaires (Structured)
 Participant`s responses to questions by researcher
 Data is usually collected by means of a format,
written document (instrument)
 Uses an interview schedule for questions that are
asked orally (face to face or via phone)
 Uses a questionnaire when participants complete
the instrument themselves.
 Close-ended questions (fixed alternative
Likert Scale (Summated Rating Scales)
questions)
 Response alternatives are specified by
the researcher
 Ensures comparability of responses
 Facilitates analysis
 Difficult to develop
 Could lead to overlooking something
important
 Open-ended questions
 Allows participants to respond to
questions in their own words
 Allows for richer, fuller information
 Instrument construction
 Develop outline of content of research
 Design questions
 Pretest
- Trial run to determine if instrument is free of
biases, errors, etc.
 Advantages of questionnaires
 Less costly
 Require less time and effort to
administer
 Can be completely anonymous
 No biases relating to the researcher
being present
 Advantages of Interviews
 Response rate is higher in face-to-face
interviews
 Effective for those that cannot
complete questionnaires (children,
blind, ESL, elderly)
 Questions are less likely to be
misinterpreted than questionnaires
 Interviews can produce additional
information through observation
 Interviews are considered to be superior to
questionnaires
Composite Scales (Social-Psychological)
 Scale: assigns a numeric score to people to
place them on a continuum with respect to
attributes being measured.
 Types:
1. Likert Scale
2. Semantic Differential Scale
3. Visual Analog Scale
 Consists of several declarative statements that
express a viewpoint.
 Participant indicates the degree to which they
agree to disagree.
J|1
  Able to summate the scores allowing for
discrimination among people with different
viewpoints.
Semantic Differential
 Participants rate a concept on a series of bipolar
adjectives
 Can measure any concept
Visual Analog Scale
 The scale is a straight line with anchors which are
the extreme limits of experience or feeling
 Measures subjective experiences
Advantages of Scales;
 Scales allow researcher to efficiently quantify
the strength and intensities of individual
characteristics
 Discriminates among people with different
attitudes, fears, motives, perceptions,
personality traits, needs.
 Good for group and individual comparisons
 Can be implemented either verbally or in
writing
Disadvantages of Scales;
Response Set Biases
 Social Desirability Response Set Bias
- Participants give answers that are common
social views
 Extreme Response Set Bias
- Participants express attitudes or feelings in
the extreme (always, never)
 Acquiescence Response Set Bias
- Participants agree with all statemets (yea-
sayers or nay-sayers)
Ways to reduce response set bias
 Countering-balancing: positively and
negatively worded statements
 Developing sensitively worded questions
 Creating a permissive, non-judgemental
atmosphere
 Guaranteeing confidentiality
iii. Vignettes
 Brief description of events or situations to
which participants are asked to react
 Information about perceptions, opinions, or
knowledge
 Questions post vignettes may be open-ended
or close-ended
 Economical to administer
 May contain response biases
iv. Projective Techniques
 Verbal self reports to obtain psychological
measurements
 Seek minimal participants` conscious
cooperation
 Ambigious or unstructured stimuli elicits
participants needs, motives, attitudes,
personality traits
 i.e., Inkblot test, word association, role playing,
drawing
 Useful in children, hearing or speech impaired
v. Q Sorts
 Uses a set of card with words, phrases or
statements
 Participant sorts cards along a bipolar
dimension (agree/disagree)
Advantages of Self-Reporting Methods;
Most common method of data collection used by
nurses
Reveal information that is difficult to obtain by
other means
Can gather retrospective and prospective data
Can measure psychological characteristics
Disadvantages of Self-Reporting Methods;
Questionable validity and accuracy
Biases
b. Observation
Observational Methods
 An alternative to self-reports
 Can be used to gather information such as
characteristics, condition of individuals, verbal
communication, nonverbal communication,
activities, and environmental conditions.
 Researcher has flexibility in the following areas;
 The focus of observation
- What events are to be observed
 Concealment
J|2
  Duration of observation
 Method of recording observations
 Types;
1. Categories and checklists
2. Rating scales
Category System
 Attempts to designate information in a systematic,
quantitative manner
 Clear definition of behaviors and characteristics to
be observed is necessary
 Lists all behaviors or activities the observer wants
to observe and records occurrences
Checklist – instrument to record observations.
Rating Scales – are tools that requires the observer to
rate some phenomena along a descriptive continuum.
 Observational Sampling
1. Time Sampling – selection of time periods
for observations
2. Event Sampling – selects behaviors or
events for observation.
Evaluation of Observational Methods;
a. Advantages;
Provides depth and variety of information
Some problems are better suited to
observation
b. Disadvantages;
Potential ethical issues
Lack of consent to be observed
Participants reaction to be observed
Biases
- Faulty inferences
c. Biophysiologic measures
Փ Types;
1. In vivo
- Measures performed directly within or on
living organisms
- i.e., blood pressure, temperature
2. in vitro
- data gathered from participants by extracting
some biophysiologic material from them for
lab analysis
- i.e., blood work, microbiologic measures,
cytology and histological measures
Advantages;
 Are relatively accurate and precise
 Are objective
 Provide valid measures of targeted variables
 Equipment is readily available
Disadvantages;
 Measuring tool may affect variables it is
attempting to measure
 Interferences may create artifact
 Energy must often be applied to the organism
when taking measurements
II. MEASUREMENT AND ASSESSMENT OF DATA
Measurement
⌂ Involves rules for assigning numeric values to
qualities
⌂ Determines how much of an attribute is present
⌂ Quantification – communicates the amount in
numbers.
Advantages;
 Removes guesswork in gathering information
 Tends to be objective
 Obtains precise information
 Can differentiate among people who possess
different degrees of an attribute
 Common language
Errors of Measurement;
◊ Always the potential for error in all tools
◊ Extraneous factors affect measurement and
distort results
 Obtained Score – is observed score
 True Score – true score if no errors
 Error of Measurement – the different
between the true and obtained scores
Factors contributing to Errors of Measurement:
1) Situational contaminants – people`s awareness of
observer, environmental factors
2) Response set biases
3) Transitory personal factors – fatigue, mood,
hunger (temporary)
4) Administration variations – alterations in data
collection methods
J|3
 5) Item sampling – errors introduced as a result of  If sample too homogenous, the lower reliability
sampling coefficient will be (instruments are designed to
measure differences)

Reliability of Measuring Instruments;

 If a measuring device is not reliable, it cannot be
RELIABILITY valid.
 Refers to the consistency with which an  High reliability of an instrument provides no
instrument measures the attribute. evidence of its validity
 The less variation in repeat measures the higher  Low reliability is evidence of low validity
its reliability.  An instrument can be reliable without being valid.
 Aspects;  Reliability consistently measures accurately,
 Stability Validity measures what it is supposed to do.
 Internal consistency
 Equivalence

Stability Validity of Measuring Instruments;

 The extent to which the same scores are VALIDITY
obtained when the instrument is used with the  Is the concern whether the measurement tools
same people on separate occasions. actually measure what they are supposed to
 To assess stability: Test-retest Reliability measure.
 Researcher administers the same measure to a  Aspects;
sample of people on two occasions and then 1. Face Validity
compares the scores. 2. Content Validity
3. Criterion-related Validity
Internal Consistency 4. Construct Validity
 Reliable to the extent that all its subparts
measure the same characteristics. Face Validity
 To assess internal consistency: Split-half  Whether the instrument looks as though it is
Technique measuring the appropriate construct.
 The items comprising the rest or scale are split
into two groups and scored, compute reliability Content Validity
coefficient.  Concerned with adequacy of coverage of the
content area being measured.
Equivalence  Tests of knowledge
 Determines the consistency or equivalence of  Psychosocial traits
the instrument by different observers or raters.  Based on judgment
 To assess equivalence: Interrater (interobserver)
Reliability Criterion-related Validity
 Has two or more trained observers make  Wants to establish the relationship between
simultaneous, independent observations, score on an instrument and some external
compete reliability coefficient. criterion.
 Compute a validity coefficient – correlates
scores on the instrument with scores in
Reliability Coefficients – a quantitative statistic that the criterion variable.
estimates how reliable an instrument is.
 Determine an instrument`s quality Construct Validity
 Low reliability makes it difficult to adequate test  Concerned with what construct is the instrument
research hypothesis actually measuring.
 To assess construct validity:
 Known-groups Technique – groups that
are expected to differ on certain
J|4
 attributes are administered the
instrument then scores re compared.
 Factor Analysis – statistical procedure
 Examination of relationships based on
theoretical predictions.
III. DATA COLLECTION QUALITATIVE RESEARCH
(Question for thought: What are the systematic rules
for analyzing qualitative data?)
a. Self-Report Methods (Unstructured)
1. Interviews
2. Diaries
3. Observation
Types of Self-Reports (Unstructured);
1) Interview
- Flexible
- Not directed by set questions
- Interviews are conversational in nature
- Usually interviews are long
- Can be tape recorded or researcher may take
notes
Completely Unstructured Interviews
 Start with broad (grand tour) questions
 Further questions are guided by initial responses
– one question`s answer leads to the next
question
Focused or Semi-Structured Interviews
 Researcher lists topics that must be covered in
an interview
 Uses a topic guide to ensure all question areas
are covered.
Focus-group Interviews
 Interviews with groups of 5-15 people whose
opinions and experiences are solicited
simultaneously
 Uses topic guide to guide questions.
2) Life Histories
◊ Narrative self-disclosures about life
experiences
◊ Has informants describe experiences in
chronological order
◊ Orally or written
3) Diaries
◊ Have informants maintain daily logs of some
aspect of their lives.
b. Observational Methods
 Unstructured Observation
 Attempt to see the world as the
participants see it
 Participant observation – data collector
actually participates in the group
o Participation can be from the role as
an observer or totally immersed in
the social setting as a participant
o Researcher needs to gain entrée
into the social group under
investigation
o Researcher needs to establish
rapport and develop trust within
the group
 Observational Data Collection
a. Physical Setting – in what context is the
human behavior occurring.
b. Participants – information about the
participants, what are their roles,
characteristics
c. Activities – what are the participants doing
d. Frequency and Duration – specific
information about the activity
e. Process – how is the event occurring
f. Outcomes – why is the activity occurring
and what are the results
g. Single Positioning – staying in a single
location
h. Multiple Positioning – involves moving
around to observe behavior from different
perspective
i. Mobile Positioning – involves following a
person throughout a given activity
 Observation Data Recording
 Uses logs and field notes
a. Log – records daily events
b. Field notes – observer`s efforts to
record information and understand
data
c. Observational Notes – descriptions of
events and conversations
J|5
 d. Theoretical notes – interpretive
attempts to attach meaning to
observations
e. Methodologic notes – instructions
about what conversations that need to
be made
f. Personal notes – comments about
researcher`s own feelings
Assessment of Qualitative Data;
⌂ Do the measures used by the researcher yield data
reflecting the truth
⌂ Qualitative research attempts to do this through
establishing the data`s trustworthiness
Establish Trustworthiness by assessing;
1. Credibility
2. Dependability
3. Confirmability
4. Transferability
a. Credibility
 Confidence in the truth of the data
 Prolonged engagement and persistent
observation
- Sufficient time to collect data, focus on the
phenomena being studied
 Searching for Disconfirming Evidence – search
for data that challenges the emerging
conceptualization or theory
 Researcher Credibility
b. Dependability
 Data stability over time and over conditions
 Stepwise Replication – having several
researchers break into teams and evaluate the
data separately and then compare conclusions
 Inquiry Audit – scrutiny of the data and
supporting documents by an external reviewer
c. Confirmability
 The objectivity or neutrality of the data, can
other independent people agree about data`s
relevance
 Audit Trail – documentation that allows an
independent auditor to come to the same
conclusions about the data
d. Transferability
 The extent to which the findings from the data
can be transferred to other settings or groups.

TRIANGULATION
 Use of multiple referents to draw conclusions,
attempts to distinguish true information from
errors
 Data source Triangulation – multiple data
sources (interviewing diverse informants on
same topic)
 Investigator Triangulation – using more than
one person to collect data
 Theory Triangulation – using multiple
perspectives to interpret data
 Method Triangulation – using multiple methods
(observation and interviews)
 External Check: Peer debriefing and member
checks
o Peer Debriefing – review and explore
various aspects of inquiry with objective
peers.
o Member Checks – providing feedback to
study participants and assessing their
reactions.

J|6