You are on page 1of 15

Experiments

Uniquely suited to identify cause-effect relationships To study effect of one variable (treatment) on another (outcome/dependent variable) Use a control group to rule out other causes Program is the treatment in a program evaluation; desired outcomes are effect Measure change with vs without the program, not just before vs after

Uses of Experiments in PRTR


Effects of information or promotion programs
on knowledge, attitudes, or behavior.

Consumer response to marketing mix changes


price, product, promotion, place

Effectiveness of various TR interventions Impacts of tourism on community/region

community attitudes, social, economic, and environmental impacts.

Benefits/Effects of recreation and tourism activity physical health, mental health, family bonding, economic impacts, learning, etc.. Studying preferences
for landscapes and more generally to measure the relative importance of different product attributes in consumer choices. e.g. conjoint analysis

Characteristics of a true Experiment


1. Sample equivalent experimental and control groups
2. Isolate and control the treatment

3. Measure the effect

Pre-test/Post-test with Control


R
R

MB1
MB2

MA1 Experimental group


MA2 Control group

R denotes random assignment to groups X denotes the treatment

Measure of effect = = =

Expmt gp - Control gp
(MA1-MB1) - (MA2-MB2) with vs without

Example
Expmt Control Pre 75% 70% Post 90% 80%

Effect

= (90-75) - (80-70) = 15% - 10% = 5%

With vs without the treatment = 5% Before vs After = 15%

Threats to Internal validity


* Pre-measurement (Testing) : effect of premeasurement on dependent variable (post-test)

* Selection: nonequivalent experimental & control groups,


(statistical regression a special case)

* History: impact of any other events between pre- and


post measures on dependent variable

* Interaction: alteration of the effect due to interaction


between treatment & pre-test.

Maturation: aging of subjects or measurement procedures Instrumentation: changes in instruments between pre and
post.

Mortality: loss of some subjects

Threats to external validity

Reactive error - Hawthorne effect - artificiality of


experimental situation

Measurement timing - measure dependent variable at


wrong time, miss effect.

Surrogate situation: using population, treatment or


situation different from real one.

Quasi-experimental designs

Ex post facto (after the fact) No control group Subjects self-select to be in expmt group
1. Travel Bureau compares travel inquiries in 1991 and 1994 to evaluate 1992 promotion efforts. 2. To assess effectiveness of an interpretive exhibit, visitors leaving park are asked if they saw exhibit or not, Two groups are compared relative to knowledge, attitudes etc.

Lab vs Field Experiments Internal vs External Validity


Internal validity - are findings correct for the
particular subjects & setting

External validity - can we generalize results


to other similar situations/populations?

Lab Expmt: high internal validity, low external Field Expmt: high external validity, low
internal

Ad Evaluation -Woodside Example


Design: 30,000 magazine subscribers, randomly assign 10K to each of three groups A, B and C. Treatments: 2 expmtl groups, 1 control
A- fun in sun message B relax with family message C no ad , control group

Measures of effect:
Total inquiries received Unaided ad recall via phone survey of 3,000 subscribers Expenditures of predicted visitors from each group (phone survey)

Results
Measure of effect
Inquiries/1000 subscribers
Unaided awareness of destin. Party visits/1000 subscribers

Spending per trip


Total spending/1,000 subsc. Net tax revenue (10%) per K Ad costs / 1,000 subscribers Net tax/ ad $ (ROI) Tax revenue/ad cost

A B 30 10 12% 4% 9.0 2.0 $400 $400 $3,600 $ 800 $360 $80 $40 $40 $9.0 $2.0 $320 $40

C 5 2% .5 $200 $ 100 $10 $0 $10

Recommendations
A-B-C Copy Split Large sample sizes 1,000 plus Compare alternatives with each other and to no ad - A to B and A/B to C Track multiple measures of impact/effect Gather spending to estimate ROI

Pricing Expmt- Bamford/Manning


Design: Vary campsite pricing for prime campsites at Vermont State Treatments: Price differentials of $1-$5
Assign state parks to treatment groups

Measures of effect:
Percent choosing prime sites Campsite occupancy shift index (compare with previous year) Revenue generated Equity acceptance of policy, income group differences

Pricing Expmt Results


Occupancy shift of 5% for each $1 differential
Pct choosing prime = 54 - .5* Pctage Price Increase E.g. $ 0 differential 54% choose prime 10% differential - 49% ; 20% diff - 44%

Revenue increase of 4 -22% Small differences in income groups


Pct choose prime 20% for L, 25% M 26% H Fee Fair? 49% L, 51% M , 60% H