Research Design

The Research Process

Step 1: Defining the Problem

Step 2: Developing an Approach to the Problem

Step 3: Formulating a Research Design

Step 4: Doing Field Work or Collecting Data

Step 5: Preparing and Analyzing Data

Step 6: Preparing and Presenting the Report

 Research Design

• A master plan that specifies the methods and

procedures for collecting and analyzing needed
information.
 Tasks Involved In a Research Design
Figure 3.8
Tasks
Involved

Define the Information Needed

in a
Research
Design

Design the Exploratory, Descriptive, and/or

Causal Phases of the Research
Today’s
Specify the Measurement and Scaling Topic
Procedures

Construct a Questionnaire

Specify the Sampling Process and the Sample

Size

Develop a Plan of Data Analysis

 A Classification of Market Research Designs

Research
Design
Exploratory Conclusive
Research Research

Experience Pilot Case

Surveys Studies Studies

Secondary
Data
 A Classification of Market Research Designs

Research
Design
Exploratory Conclusive
Research Research

Cross-sectional
Study
Descriptive Causal
Longitudinal Design Design
Study how it
related to -
Experiment
Secondary Survey Observation
Data Study
 Types of Research Designs

Exploratory research to gain ideas and insights

 Newspaper facing decreasing sales to generate possible
explanation.

Descriptive research to obtain summary measures to address

research questions (research objectives are clearly defined).
 Trends in lifestyle with respect to age, sex, etc.

Causal research for cause-and-effect connection between

managerial decisions and market outcome.
– How people react to a newspaper’s topic selection and
space allocation.
 Exploratory & Conclusive Research Differences

Exploratory Conclusive
Objective: To provide insights and To test specific hypotheses and
understanding. examine relationships.

Character- Information needed is defined only Information needed is clearly defined.

istics: loosely. Research process is flexible Research process is formal and
and unstructured. Sample is small structured. Sample is large and
and non-representative. Analysis representative. Data analysis is
of primary data is qualitative. quantitative.

Findings Tentative. Conclusive.

/Results:

Outcome: Generally followed by further Findings used as input into decision

exploratory or conclusive research. making.
 A Comparison of Basic Research Designs

Exploratory Descriptive Causal

Objective: Discovery of ideas Describe market Determine cause and
and insights characteristics or effect relationships
functions
Manipulation of one
Characteristics: Flexible, versatile Marked by the prior or more independent
formulation of specific variables
hypotheses
Control of other
Often the front end Preplanned and mediating variables
of total research structured design
design Experiments

Expert surveys Secondary data

Methods: Pilot surveys Surveys
Secondary data Panels
Qualitative research Observation and other
data
 Exploratory Research

• Usually conducted during the initial stage of the research

process
• Purposes
– To narrow the scope of the research topic, and
– To transform ambiguous problems into well-defined ones
 Exploratory Research Techniques

• Secondary Data Analysis

– Secondary data are data previously collected & assembled for
some project other than the one at hand
• Pilot Studies
– A collective term for any small-scale exploratory research
technique that uses sampling but does not apply rigorous
standards
– Includes
• Focus Group Interviews
– Unstructured, free-flowing interview with a small group of people
• Projective Techniques
– Indirect means of questioning that enables a respondent to project beliefs
and feelings onto a third party or an inanimate object
– Word association tests, sentence completion tests, role playing
 Exploratory Research Techniques

• Case Studies
– Intensively investigate one or a few situations similar to the
problem situation
• Experience Surveys
– Individuals who are knowledge about a particular research
problem are questioned
 Conclusive Research

• Provide specific information that aids the decision maker in

evaluating alternative courses of action
• Sound statistical methods & formal research methodologies are
used to increase the reliability of the information
• Data sought tends to be specific & decisive
• Also more structured & formal than exploratory data
 Types of Conclusive Research

• Descriptive Research
– Describes attitudes, perceptions, characteristics, activities
and situations.
– Examines who, what, when, where, why, & how questions
• Causal Research
– Provides evidence that a cause-and-effect relationship exists
or does not exist.
– Premise is that something (and independent variable)
directly influences the behavior of something else (the
dependent variable).
 Cross Sectional vs. Longitudinal Designs
Figur
e 3.6
Cross
Sectio
nal
vs.
Longit
udinal
Desig
ns

Cross Sample
Sectional Surveyed
Design at T1

Same
Sample Sample also
Longitudinal Surveyed at
Design Surveyed at
T1 T2

Time T1 T2
 Cross-Sectional vs. Longitudinal Designs

Cross-Sectional Longitudinal

Detecting change Worse Better

Amount of data collected Worse Better

Representativeness Better Worse

Response bias Better Worse

 Common Characteristics of Causal Studies

• Logical Time Sequence

– For causality to exist, the cause must either precede
or occur simultaneously with the effect
• Concomitant Variation
– Extent to which the cause and effect vary together
as hypothesized
• Control for Other Possible Causal Factors
 How Descriptive & Causal Designs Differ

• Relationship between the variables

– Descriptive designs determine degree of association
– Causal designs infer whether one or more variables influence
another variable
• Degree of environmental control
– Descriptive designs enjoy lesser degrees of control
 Descriptive Research
1. Surveys

• May be used to reveal • May be used to

summary statistics by explore relationships
showing responses to between 2 or more
all possible variables.
questionnaire items.
• Often provide leads in
identifying needed
changes
 Descriptive Research
Survey Forms

• Written questionnaires Factors to be considered

• Sampling
• Type of population
• Personal interviews • Question Form
• Telephone interviews • Question Content
• Response rates
• Costs
• Available facilities
• Length of data collection
• Computer assisted
techniques for data
collection
 Descriptive Research
2. Survey Form - Interviews

• More time efficient

• Allow the researcher to establish a rapport
with the respondent
• Allow the acquisition of more in-depth
information
• Allow for interviewer observation
• Allow the interviewer to obtain visual cues
• May be personal or telephone interviews
 Descriptive Research
Survey Form – Personal Interviews

Disadvantages
• Require more staff time
• Require more travel time
 Descriptive Research
Survey Form – Telephone Interview

Advantages Disadvantages
 Less expensive  Limited telephone
 Less time-consuming access
 Lack of interviewer’s
ability to observe the
respondent and
obtain visual cues
 Descriptive Research
Survey Form – Mailed Questionnaires

Advantages Disadvantages
 Ability to reach large  Lower response rate
number of people across
a wide geographic area  Need to design a
 Ease and low cost of survey instrument
distribution with a simple format
 Minimal amount of staff
required
 Allows respondents to
respond in their time
frame
 Descriptive Research
Survey Form – Mailed Questionnaires

• A letter of transmittal should accompany mailed

questionnaires.
• Should state purpose and importance of research
• Should state importance of responding
• Should give a time frame to respond
• Should include a confidentiality statement
• Should include an offer to share results
• Should include a thank-you note to the respondent
 Descriptive Research
Characteristics of a Good Survey

• Good questioning techniques

• Use complete sentences
• Offer a limited set of answers
• Interesting
• Worded so that questions mean the same to all
• Provide definitions for confusing terms
• Uses the “I don’t know” answer very carefully
 Descriptive Research
3. Observational Research Methods

1. Naturally occurring behaviors observed in

natural contexts
2. Contexts that are contrived to be realistic
 Descriptive Research
Observational Research Methods

• Require direct observation of behavior

• Data gathered without intermediary
instruments
• Can yield a wealth of invaluable information
• Can be a complicated process
 Descriptive Research
Observational Research Methods

• Can be employed productively to support

many purposes in educational technology
• Can be used to determine how people interact
with technology in various stages of design
and implementation
 Descriptive Research
Observational Research Methods
• 2 Forms of Observational Research
– Structured
– Unstructured
 Descriptive Research
Observational Research Methods
Structured Observations Unstructured
Rigid and controlled Observations
 Used to determine
Predetermined unselective, detailed,
methods continuous description of
behavior.
 Detects unintended effects
 More time consuming
because of time and labor
required to collect and
analyze sets of extensive
observations
 Descriptive Research
Observational Research Methods
Develop observation form
May be paper and pencil or electronic
May use a rating scale to evaluate behavior
A 3-point rating scale is sufficient
 Descriptive Research
Observational Research Methods
Newer Mediated Observation Techniques
• Audio
• Videotape
• Computers provide on-line monitoring (process of
capturing characteristics of the human-computer
interaction automatically)
– Keystroke records
– Logging data
 Impact and Future
of Descriptive Research
• Descriptive Research methods have gained
acceptance
• Number of descriptive studies published in
research journals has increased
• Descriptive research leads to prescriptions
that instructional designers and educators can
heed as they consider future direction
 Questions Addressed by Causal Research
* Marketing director of local beer company,
“Will replacing TV commercial A with commercial B lead
to increase in consumer preference on our brand?”
* Chairman of a charity organization. “Will it be
worthwhile to mail to previous donors an attractive and
expensive brochure to solicit higher contributions this
year?”
* The sales manager of a local life insurance company.
“Will training in the use of computers for client
management increase agents’ sales?”
* Marketing VP of fashion chain, “Can we improve
profitability of our fashion clothing line by increasing its
price by 10%?”
 Causality: Cause-and-Effect
Change-in-X causes change-in-Y

Evidence of Causality
• Concomitant variation:
– If X changes, then Y also changes.
– If X does not change, then Y does not
change.
• Time order: cause (X) occurs before effect (Y).
 Types of Experiments
Laboratory experiment
Research investigation in which investigator
creates a situation with exact conditions so as
to control some, and manipulate other,
variables

Scientific investigation in which an

investigator manipulates and controls
one or more independent variables and
observes the dependent variable for
variation concomitant to the manipulation
of the independent variables Field experiment
Research study in a realistic situation in
which one or more independent variables are
manipulated by the experimenter under as
carefully controlled conditions as the
situation will permit