Unit 3

Research Design
 Research Design
 It is like a blueprint for the collection, measurement and analysis of
the data
 A conceptual structure within which research is going to be carried
out

Elements of Research Design:

 The Purpose of Study (Exploratory/Descriptive/Causal)

 Extent of Researcher Interference

 Study Settings (Location)

 Research Strategies (Experiments/Survey/Interviews/ Case Studies)

 Unit of Analysis (Level of data analysis)

 Time Horizon
The Research Design (Cont’d)
The Purpose of Study (Types Of
Research)

 Exploratory
 Descriptive
 Causal
Uncertainty Influences the type of Research

CAUSAL OR COMPLETELY ABSOLUTE EXPLORATORY

DESCRIPTIVE CERTAIN AMBIGUITY
 Degree of Problem Definition
Exploratory Research
(Unaware of Problem)
“Our sales are declining and
we don’t know why.”
“Would people be interested
in our new product idea?”
Descriptive Research Causal Research
(Aware of Problem) (Problem Clearly Defined)
“What kind of people are buying “Will buyers purchase more of
our product? Who buys our our products in a new package?
competitor’s product?”
“Which of two advertising
“What features do buyers prefer campaigns is more effective?”
in our product?”
7
Exploratory Research

Secondary data

Experience survey

Pilot studies
 The Purpose of Study (Type of Research)

Exploratory study:

 Carried out when not much is known about the problem

 No details are available on how similar problems have been solved in the
past
 Extensive preliminary work needs to be done
 Major emphasis on the discovery of ideas and insights
 Often relies on secondary data and/or qualitative approaches (informal
discussion with managers/employee/consumers, formal approaches like
interviews/focus groups/case studies/projective methods)
 Does not provide conclusive evidence
 Subsequent research expected
Descriptive study:

I keep six honest serving men, (they taught me all I

knew), their names are

what, and why, and when, and how,

and where and who.”
--Rudyard Kipling
Descriptive study:

 Describes characteristics of a population or phenomenon

 Often designed to collect data and establish relationship describing

persons/events/situations

 Can be either quantitative (production figures/sales figures/demographic

data)or qualitative (consumers decision making process)

 Can be done using survey or case study methods

Descriptive Research Example

 Woman about 40 years old

 Household income of about $50,000 p.a.
 Woman having at least some college education
Descriptive Research Findings Example

 Men’s fragrance market is1/3 size of women’s fragrance

market but growing at a faster pace

 Women buy 80 % of men’s fragrances

 Survey Method

 A survey is defined as a method of gathering primary data based on communication with a

representative sample of individuals

 Surveys require asking respondents, for information, using either verbal or written questions

 Questionnaires or interviews are utilized to collect data on the telephone, face-to-

face, and through other communication media
 Steps in Conducting a Survey

 Develop the hypothesis; decide on type of survey (mail, interview, telephone);

 Write survey questions (decide on response categories, design lay out)
 Plan how to record data; pilot test survey instrument
 Decide on target population; get sampling frame; decide on sample size; select the sample
 Locate respondents; conduct interviews; carefully record data
 Recheck all data; perform statistical analysis on data
 Describe methods and findings in research report
 Present findings to others for critique and evaluation
Case Study Method

 A case study is a report about a specific object, event or activity

 Case might be an individual, a group, an organization, an event or a situation
 Allows for exploration and understanding of complex issues
 Examine the real life situation from different angles
 Multiple methods of data collection are used
 A robust method particularly when a holistic, in-depth investigation is required
 Researcher is able to go beyond the quantitative statistical results and
understand the behavioral conditions through the actor’s perspective
 Include both quantitative and qualitative data
 Helps explaining both the process and outcome of a phenomenon
 Complete observation, reconstruction and analysis of the case
Causal Research

 Conducted to identify cause and effect relationships

Explanatory or Causal study:

 It is the heart of the scientific approach

 Test whether or not one variable causes another to change

 Focuses on delineating one or more factors causing problem

Following four conditions must be satisfied:

 The independent and dependent variables should covary

 The independent variable should precede dependent variable
 No other factor should be a possible cause of change in the dependent variable
 A logical explanation is must for establishing the relationship
Problem Discovery Problem Selection of
and Definition discovery exploratory research
technique
Sampling

Selection of
exploratory research
technique Probability Nonprobability

Secondary
Experience Pilot Case Collection of
(historical) Data
survey study study data
data Gathering
(fieldwork)

Data
Editing and
Problem definition Processing
coding
(statement of and
Analysis data
research objectives)

Data
Selection of processing
Research Design basic research
method Conclusions
Interpretation
and Report
of
findings
Experiment Survey
Secondary
Laboratory Field Interview Questionnaire Observation
Data Study Report
Extent of Researcher Interference

 Minimal Interference

 Moderate Interference

 Excessive Interference
Study Setting: Contrived and Non-Contrived

 Research conducted in the natural environment (non-contrived settings)

 Cause and effect relationship (employees/managers/consumers) under natural

environment are called as field experiments

 Research conducted in an artificial environment (contrived settings),e.g. causal

studies

 Cause and effect relationship (employees/managers/consumers) under

controlled environment are called as lab experiments
 Time Horizon

 Cross Sectional Studies (One-Shot Studies):

Data are gathered just once over a defined period of time.
e.g. data gathered from stock brokers between April and June to study their
concerns in a turbulent stock market.

 Longitudinal Studies
Data are gathered at more than one point of time.
Purpose is to examine continuity of response and to observe changes that occur
over time
e.g. Data gathered for studying employees behavior before and after change in top
management.
e.g. Skill development before and after MIP
 Experimental Research Designs

What is an Experiment?

 Used to infer causality

 Researcher manipulates one or more causal variables and then measures the effect
on the dependent variable of interest

 Since any changes in the dependent variable may be caused by a number of other
variables, the causal relationship often tends to be probabilistic.
Experimental Group versus Control Group
Necessary Conditions for making Causal Inferences

1. Concomitant Variation
Extent to which cause X and effect Y vary together

2. Time order of occurrence of variables

Causal variable must occur prior to or simultaneously with the effect variable

3. Absence of other possible causal factors

Concepts used in Experiments

 Independent Variables

 Test Units

 Dependent variables

 Experiment

 Extraneous Variables
Essentials for conducting an Experiment

 Treatments to be manipulated (independent variables)

 Test units to be used

 Dependent variables to be measured

 Procedure for dealing with the extraneous variables

Two goals of researcher while conducting
an Experiment

 To draw valid conclusions about the effect of treatments

(independent variable) on the dependent variable (concerned
with internal validity)

 To make generalization about the results to a wider population

(concerned with external validity)
Validity of experiments

 If a study is valid then it truly represents what it was intended to represent.

 Experimental validity refers to the manner in which variables that influence both
the results of the research and the generalizability to the population at large.

 It is broken down into two groups:

(1) Internal Validity and
(2) External Validity
Internal Validity
 It tries to examine whether the observed effect on a dependent
variable is actually cause by the treatments (independent
variables)

 For an experiment to be internally valid, all other causal factors

should be absent

 It is a basic minimum-must be present

 Impossible to draw causal relationship under the influence of

extraneous variables

 Hence control of extraneous variables is a must

 Without internal validity, experiment gets confounded

External Validity

 It refers to the generalization of the experimental results

 It is desired to have both internal and external validity in an experiment

 However, in real time scenario, a trade-off between the two is made

 To remove the influence of extraneous variable, an experiment is set up in

artificial setting, there by increasing its internal validity

 However, in the process the external validity will be reduced

Definition of Symbols

 X = The exposure of a test group to an experimental treatment whose effect is to

be measured

 O = The measurement or observation of the dependent variable

 R = The random assignment of test units or groups to separate treatments

Some Conventions

 Movement from left to right indicates the time sequence of events

 All symbols in one row indicate that the subject belongs to that specific
treatment group

 Vertical arrangement of the symbols means that these symbols refer to the
events or activities that occur simultaneously
Example 1
Consider the following symbolic arrangement:
O1 X O2 O3

 There is one group whose members were not selected randomly

 The group of test unit was exposed to treatment X

 The measurement (O1) on the group was taken prior to applying treatment X

 Two measurements (O2, O3) on the group were taken after the application of the
treatment at different points of time
Example 2

 Consider the symbolic arrangement:

R O1 X O2
R X O3
Factors affecting internal validity of
the experiment
1. History:

It refers to those specific events that are external to the experiment but
occur at the same time as the experiment.
For e.g. O1 X O2

 Here the treatment effect (O2 - O1) (if positive) can’t be attributed to
sales training only
 It may also because of improved general economic condition between
O1 and O2
 Hence, higher the time difference between the two observations, higher
the chances of history confounding an experiment
2. Maturation

 It is similar to history

 Concerned with the changes in a test unit occurring with the passage of time

 These changes are not due to the impact of treatments

For e.g.
People getting older, more experienced, more matured; tired, uninterested (in
case of decline)

 Again longer the time difference between the two observations, higher the
chances of maturation confounding an experiment
3. Testing

 Concerned with the possible effect on the experiment of taking a

measurement on the dependent variable before presentation of the
treatment

 Testing effects are of two kinds:

 Main testing effect : occurs when the first observation influence the second
 Reactive or interactive testing effect: occurs when both the observations
influence each other

 For e.g. if sales people know that they are being sent for training to
know its effectiveness, they would become sensitized and behave
differently
4. Instrumentation

 Refers to the effect caused by the changes in measuring instrument used for
taking an observation
For e.g. presenting a pre-test and post test questionnaire in a different
fashion
5. Statistical regression

 Occurs when the test units with extreme scores (extremely favourable or
unfavourable) are chosen for exposure to the treatment

 The effect is that test units with extreme scores tend to move towards an average
score with the passage of time

 This is because test units with extreme scores have more room to change

 An increase in sales after the training programme may be attributed to the

regression effect
6. Selection Bias

 Refers to improper assignments of test units to treatments.

 Test units may be assigned to the groups in such a way that the groups differ on
the dependent variable prior to the presentation of the treatment

 It occurs if test units self-select their groups

 The selection of test units to the treatment group should be random

7. Test unit mortality

 Occurs when some of the test units dropout from the experiment while it is in
progress

 There is no way finding out whether those who left would have produced the
same results as those who completed the program

 All these factors ( extraneous variables) discussed above are not mutually
exclusive

 They can occur together and interact with each other

 Our objective is to eliminate them as far as possible

Factors affecting External Validity

 The environment at the time of test may be different from the environment of
the real world where these results are generalised

 Population used for experimentation of the test may not be similar to the
population where the results of experiment are to be applied

 Results obtained in a 5-6 weeks test may not hold for 12 months

 Treatment at the time of test may be different form the treatment to the real
world
 E.g. Can the results of the success of a product in a test market be extrapolated
to the entire nation/state/district??
Methods to Control Extraneous Variable

1. Randomization:

 Refers to the random assignments of test units to the experimental groups

 Treatments are also randomly assigned to the experimental groups
 However, for randomization to be effective, a large sample size is required

2. Matching:

 Match the various groups by confounding variables (for e.g. gender, income,
age, work exp.)
 It may not be possible to match all the confounding variables
3. Use of experimental designs

4. Statistical Control

 If all the above discussed methods fail to eliminate the extraneous variable, then
the experiment gets confounded and it is not possible to make causal inferences

 The effect of these variables on dependent variable can be statistically controlled

using ANCOVA (Analysis of Covariance)
Experimental Designs

 A blueprint of the procedure that enables the researcher to test his hypothesis
by reaching valid conclusions about relationships between independent and
dependent variable

 It refers to the conceptual framework within which the experiment is

conducted
Classification of Experimental Design
Ex post facto designs

 Ex post facto means from what is done afterwards

 Subjects already exposed to stimulus

 Since the study doesn’t immediately follow the stimulus, called as ex post facto design

 It is an experimental study examining how an independent variable, present prior to the

study in the participants, affects a dependent variable

 Ideal for situations where it is not possible to manipulate the characteristics of human
participants

 Substitute for true experimental research

 The researcher can observe the independent variables after the event