WEB APPENDIX JMR-14-117.

R3

A Dynamic Model for Digital Advertising:

The Effects of Creative Formats, Message Content and Targeting on Engagement

This Web Appendix contains the following sections:

A. Assessing Instrumental Variables

B. Deriving an Approximate Optimal Importance Function

C. Analysis of Data Spike

D. Findings from Alternative GLMs and Linear Models

E. Optimal Reallocations

1
Web Appendix A -- Assessing Instrumental Variables

We conducted analyses to assess the strength of our instrumental variables. First, weakness would imply
that advertising is poorly correlated with these instruments. Thus we obtained the population R-squares
from the regression of log(1+ad impression) against them. The results are relatively high within the four
targets, ranging from 0.63-.73 (below Table Q1). Weakness would also imply that variations attributable
to the instruments would be small relative to variations in ad impressions; in this case, the effects of ads
() is expected to be highly correlated with the random error in the instrumental variable equation, and the
covariance of that error with the error in advertising impression (goodwill) equation (See Rossi and
Allenby, 2005). We calculated the correlation between  and the ratio of this variance to covariance using
the posterior draws, and reported these in Table Q1. Note they are relatively small and several are not
significant, providing further evidence that our instruments can help identify ad effects.

Table Q1: IV Assessment: Correlation of  draws with the ratio, goodwill error variance/ covariance of instrument

Digital Format Message Retargeting Male Female Age

Product -0.1602 0.0813 0.0932 0.0238

Flash (160x600) Price Offer -0.0547 0.1465 -0.1172 0.1353

Product -0.0280 -0.0766 0.0141 0.0417

Flash (300x250) Price Offer 0.0595 0.0154 0.0364 -0.1156

Product 0.0243 0.0622 0.0153 0.0261

Flash (728x90) Price Offer -0.0138 0.0709 -0.0077 -0.1633

Price Offer 0.0666 -0.0096 -0.0577 0.0346

GIF (160x600) International -0.0004 -0.1517 -0.2001 -0.1894

Price Offer 0.1075 0.0144 0.0612 0.0736

GIF (300x250) International 0.0657 0.1157 -0.1217 0.0457

Price Offer 0.1337 0.0921 0.0728 0.0579

GIF (728x90) International 0.0547 0.1958 -0.1302 0.1050

*Bold p-value less than 0.05. ** Population R-Squares: Retargeting (0.7281), Male (0.6961), Female (0.6313), Age (0.7067)

Reference:
Rossi, Peter, Greg Allenby, and Rob McCullogh (2005), Bayesian Statistics and Marketing.
Hoboken, N.J.: Wiley Series in Probability and Statistics

2
Web Appendix B -- Deriving an Approximate Optimal Importance Function

Recall that our goal is to approximate the most efficient proposal distribution for the Particle Filter, the
“optimal” importance function which has the form p( g t g t 1 , yt ) . The log of this distribution is
approximately:

B1) l ( g t )  ln p( yt g t ) p( g t g t 1 )

Given that our model is within the exponential family, l ( g t ) is twice differentiable; thus we take a second
order Taylor approximation of l ( g t ) at some point g :

1
l ( g t )  l ( g )  l ' ( g )( g t  g )  ( g t  g ) t l" ( g )( g t  g )
B2) 2

If we set ( g )  l" ( g ) 1 and m( g )  ( g )l ' ( g ) then

1
l ' ( g )( g t  g ) 
( g t  g ) t l" ( g )( g t  g )
2
B3) 1
 C  ( g t  g  m( g )) t  1 ( g )( g t  g  m( g )) , C a constant.
2

If we examine the last expression (B3), we can readily see it is a normal with a mean and variance; it thus
suggests a normal approximation for the optimal importance function:

B4) p( g t gt 1 , yt )  N m( g )  g, ( g )
~

We take an approximation at the mode g  g t* , so m( g t* )  0, since l ' ( g )  0.

Thus, an approximate optimal importance function becomes:

B5) 
p( gt gt 1, yt )  N gt* ,l" ( gt* )1
~ 
.
The mode is simulated using the Newton-Raphson algorithm.

References:

Doucet, Arnaud, Simon Godsill , Christophe Andrieu (2000), “On Sequential Monte Carlo
Sampling Methods for Bayesian Filtering,” Statistics and Computing, 10, 197-208.
Doucet, Arnaud, Nando de Freitas, and Neil Gordon, Editors (2001), Sequential Monte Carlo
Methods in Practice, New York: Springer.

3
Web Appendix C – Analysis of Data Spike

The spike potentially could overstate advertising effects, especially if ad response differs markedly during
the spike period. Thus, as a preliminary step we re-examined click-through rates at and outside the spike
in our sample. We found that click through rates at the spike was approximately 0.0550%, while average
rates after and before the spike were 0.0542% and 0.0547%, respectively; thus advertising effects might
not be substantively overstated due to the spike.

For a more formal analyses, however, we included seasonal dummies for the spike period and re-
estimated the proposed model. We note in Table R1 that the seasonal dummies are all insignificant, and
moreover our estimates remain largely unchanged, providing further evidence the spike in this sample is
unlikely to be a problem.

4
TABLE R1: ESTIMATES FROM PROPOSED MODEL: CONTROLLING FOR THE DATA SPIKE

Parameters: Retargeting Male Female Age

Flash Effect 2.5711 2.4639 2.2559 2.2405

Banner Orientation- Size:
160x600, σ -1.7712 -1.9332 -2.1737 -2.4056
300x250, σ -2.1115 -2.3316 -2.1949 -1.7058
728x90, σ -1.4173 -1.1540 -2.1381 -2.5675
Flash Formats:
Flash (160x600):
Carryover Rate, δ 0.6032 0.6759 0.5613 0.6936
Product Offer,  0.0005 0.0275 0.0401 0.0212
Price Offer,  0.1079 0.0945 0.1527 0.1184
Spike-Period 0.1326 -0.1001 0.0394 0.1433
Flash (300x250):
Carryover Rate, δ 0.7056 0.7461 0.6031 0.6822
Product Offer,  0.0010 0.0298 0.0375 0.0259
Price Offer,  0.0807 0.1031 0.1305 0.0780
Spike-Period 0.0440 -0.0796 -0.2501 0.2092
Flash (728x90):
Carryover Rate, δ 0.6886 0.7438 0.5967 0.7591
Product Offer,  -0.0099 0.0014 0.0333 0.0261
Price Offer,  0.0754 0.0353 0.1355 0.0869
Spike-Period 0.1560 -0.0577 -0.0716 0.1163
GIF Formats:
GIF (160x600):
Carryover Rate, δ 0.1439 0.1022 0.1393 0.1711
Price Offer,  -0.0448 -0.0397 0.1043 -0.0207
International,  0.7141 0.3249 0.7303 0.8337
Spike-Period -0.2095 0.1472 -0.3834 -0.1494
GIF (300x250):
Carryover Rate, δ 0.1298 0.2485 0.1555 0.2744
Price Offer,  0.0895 -0.0544 0.1130 -0.0055
International,  0.7061 0.2613 0.7289 0.6075
Spike-Period 0.1402 -0.1449 0.1961 -0.0402
GIF (728x90):
Carryover Rate, δ 0.0913 0.2109 0.1816 0.1762
Price Offer,  0.0796 -0.0289 0.1357 -0.0386
International,  0.6290 0.2756 0.7157 0.6826
Spike-Period -0.1614 0.0080 -0.0425 -0.1542
*bold 95% HPDI

5
Web Appendix D: Findings in Alternative GLMs and Linear Models

TABLE S1: LINEAR STATE SPACE MODEL NDLM – NO MEASUREMENT MODEL, NO INFLATION

Parameters: Retargeting Male Female Age

Flash Effect 14.2900 -0.8918 19.0151 6.7304

Banner Orientation- Size:
160x600, σ 0.6110 0.2529 -0.0446 2.4367
300x250, σ 0.2512 0.1228 0.1882 4.1986
728x90, σ 0.4085 0.0689 -0.0460 1.9652
Flash Formats:
Flash (160x600):
Carryover Rate, δ 0.7379 0.3126 0.1138 0.1291
Product Offer,  0.1666 0.3651 0.4943 0.6188
Price Offer,  0.2269 0.2235 0.3022 0.4646
Flash (300x250):
Carryover Rate, δ 0.7746 0.6168 0.3746 0.2457
Product Offer,  0.1108 0.1346 0.2860 0.5867
Price Offer,  0.1349 0.0906 0.1991 0.3701
Flash (728x90):
Carryover Rate, δ 0.8207 0.5257 0.2622 0.1183
Product Offer,  -0.0116 0.1062 0.2923 0.3417
Price Offer,  0.1481 0.1824 0.1955 0.5309
GIF Formats:
GIF (160x600):
Carryover Rate, δ 0.1737 0.1567 0.0159 0.0009
Price Offer,  -0.3785 -0.1341 0.4202 0.6043
International,  4.0308 0.8708 2.1279 1.9875
GIF (300x250):
Carryover Rate, δ 0.3358 0.2456 0.1085 0.0174
Price Offer,  1.6922 -0.1562 0.6248 -1.0443
International,  2.8503 0.3163 0.9974 1.1752
GIF (728x90):
Carryover Rate, δ 0.1752 0.1109 0.0262 0.0007
Price Offer,  0.2531 0.4791 0.4935 -0.4221
International,  2.4767 0.3839 1.1246 1.3328
*bold 95% HPDI, DIC = 33455.8

6
TABLE S2: LOG- LINEAR STATE SPACE MODEL NDLM – NO MEASUREMENT
MODEL, NO INFLATION

Parameters: Retargeting Male Female Age

Flash Effect 0.9613 0.1358 0.1600 1.7845

Banner Orientation- Size:
160x600, σ -0.2330 -0.2199 -0.2778 -1.2235
300x250, σ -0.4096 -0.1841 -0.3262 -1.0463
728x90, σ -0.2200 -0.0381 -0.3293 -1.2555
Flash Formats:
Flash (160x600):
Carryover Rate, δ 0.5895 0.8107 0.5099 0.7130
Product Offer,  0.0042 0.0188 0.0506 0.0220
Price Offer,  0.1177 0.0682 0.1839 0.0894
Flash (300x250):
Carryover Rate, δ 0.7000 0.7993 0.5260 0.6765
Product Offer,  0.0050 0.0155 0.0532 0.0251
Price Offer,  0.0763 0.0615 0.1698 0.0926
Flash (728x90):
Carryover Rate, δ 0.7113 0.8315 0.5434 0.7732
Product Offer,  -0.0038 0.0112 0.0472 0.0148
Price Offer,  0.0743 0.0460 0.1712 0.0671
GIF Formats:
GIF (160x600):
Carryover Rate, δ 0.0964 0.3213 0.2869 0.3411
Price Offer,  -0.0327 0.0017 0.0226 0.0279
International,  0.2065 0.0902 0.1325 0.3286
GIF (300x250):
Carryover Rate, δ 0.0519 0.0559 0.5300 0.4569
Price Offer,  0.0817 -0.0086 0.0496 0.0125
International,  0.2034 0.0706 0.0993 0.2722
GIF (728x90):
Carryover Rate, δ 0.1118 0.1501 0.4529 0.2885
Price Offer,  0.0548 0.0002 0.0460 0.0010
International,  0.1503 0.0361 0.1000 0.3374
*bold 95% HPDI

7
TABLE S3: DYNAMIC POISSON – NO MEASUREMENT MODEL, NO ZERO INFLATION

Parameters: Retargeting Male Female Age

Flash Effect 2.2248 0.9154 3.8717 2.5602

Banner Orientation- Size:
160x600, σ -1.2689 -0.8360 -4.1480 -2.1695
300x250, σ -1.8351 -1.7468 -4.1003 -1.4765
728x90, σ -1.8351 -0.3929 -4.0659 -2.4098
Flash Formats:
Flash (160x600):
Carryover Rate, δ 0.5314 0.7077 0.5248 0.6779
Product Offer,  0.0025 0.0227 0.0533 0.0292
Price Offer,  0.1242 0.0925 0.1887 0.1146
Flash (300x250):
Carryover Rate, δ 0.6878 0.8183 0.5431 0.6404
Product Offer,  0.0095 0.0329 0.0542 0.0247
Price Offer,  0.0931 0.1087 0.1671 0.0867
Flash (728x90):
Carryover Rate, δ 0.6776 0.7763 0.5476 0.5224
Product Offer,  -0.0050 0.0055 0.0483 0.3138
Price Offer,  0.0770 0.0358 0.1708 0.0936
GIF Formats:
GIF (160x600):
Carryover Rate, δ 0.0165 0.4741 0.0125 0.1097
Price Offer,  -0.0209 0.0312 0.1038 0.0242
International,  0.7977 0.2850 1.7747 0.8076
GIF (300x250):
Carryover Rate, δ 0.0214 0.4816 0.1602 0.1699
Price Offer,  0.1005 -0.0222 0.0967 -0.0031
International,  0.7866 0.3686 0.7940 0.6959
GIF (728x90):
Carryover Rate, δ 0.0877 0.2661 0.1139 0.1259
Price Offer,  0.0750 0.0060 0.1228 -0.0186
International,  0.6669 0.1653 0.9005 0.7930
*bold 95% HPDI, DIC 22774.67

8
TABLE S4: STATIC POISSON MODEL - ZERO INFLATED, NO MEASUREMENT MODEL

Parameters: Retargeting Male Female Age

Flash Effect 4.1548 1.6871 3.9437 9.3529

Banner Orientation- Size:
160x600, σ -3.3416 -1.9156 -4.4036 -5.6223
300x250, σ -3.8755 -2.4393 -4.2872 -5.0354
728x90, σ -3.1737 -1.4396 -4.2609 -5.1610
Flash Formats:
Flash (160x600):
Product Offer,  0.0065 0.0324 0.0617 0.0448
Price Offer,  0.1479 0.2235 0.2250 0.2017
Flash (300x250):
Product Offer,  0.0158 0.0328 0.0602 0.0412
Price Offer,  0.1191 0.1355 0.1990 0.2017
Flash (728x90):
Product Offer,  -0.0037 0.0113 0.0548 0.0387
Price Offer,  0.0913 0.0637 0.2007 0.1772
GIF Formats:
GIF (160x600):
Price Offer,  -0.0192 0.0161 0.0706 0.0257
International,  0.7818 0.3697 0.7390 0.9360
GIF (300x250):
Price Offer,  0.0981 -0.0589 0.1143 -0.0094
International,  0.7709 0.3849 0.6587 0.8539
GIF (728x90):
Price Offer,  0.0805 0.0194 0.1167 -0.0231
International,  0.6279 0.2212 0.6343 0.8572
*bold 95% HPDI, DIC = 25402.3

9
TABLE S5: DYNAMIC NEGATIVE BINOMIAL – NO MEASUREMENT MODEL, NO ZERO
INFLATION

Parameters: Retargeting Male Female Age

Flash Effect 3.9367 3.6984 6.4601 4.0036

Banner Orientation- Size:
160x600, σ -1.6529 -2.1392 -6.1221 -2.1718
300x250, σ -2.4792 -3.0411 -6.1534 -2.0157
728x90, σ -1.1585 -2.8274 -5.9215 -2.4633
Flash Formats:
Flash (160x600):
Carryover Rate, δ 0.2026 0.4489 0.3797 0.5487
Product Offer,  0.0049 0.0102 0.0616 0.0173
Price Offer,  0.1683 0.0881 0.2054 0.0908
Flash (300x250):
Carryover Rate, δ 0.3909 0.5554 0.3157 0.5709
Product Offer,  0.0154 0.0108 0.0701 0.0128
Price Offer,  0.1279 0.0955 0.2180 0.0716
Flash (728x90):
Carryover Rate, δ 0.3659 0.6339 0.3689 0.6699
Product Offer,  -0.0163 -0.0008 0.0543 0.0075
Price Offer,  0.0739 0.0701 0.1933 0.0580
GIF Formats:
GIF (160x600):
Carryover Rate, δ 0.0363 0.0714 0.0209 0.0520
Price Offer,  0.0059 0.0932 0.2349 0.0248
International,  1.1739 0.2850 1.8750 0.8076
GIF (300x250):
Carryover Rate, δ 0.0428 0.2231 0.0402 0.0507
Price Offer,  0.1292 0.0133 0.2140 -0.0231
International,  1.1822 0.3686 1.7785 0.6959
GIF (728x90):
Carryover Rate, δ 0.0990 0.0431 0.0374 0.0679
Price Offer,  0.0910 0.0185 0.2407 -0.0296
International,  0.8979 0.1653 1.6930 0.7930
*bold 95% HPDI, DIC 31737.22

10
Web Appendix E -- Optimal Reallocations

We optimally re-allocated impressions for one, two and four successive periods, given the information
available at t-1. Table T1 reports the % change in impressions in each case.

Table T1: ALLOCATIONS -- Actual Impressions (1.0e+8) and Model Based Allocation Changes (%)
Actual
Digital Format Exposure Impressions %(t-1|t)(1) %(t-1|t))(2) % (t-1|t)(4)

Product 2.3783 -3.0676 -2.9570 -4.8998

Flash (160x600) Price Offer 3.3685 34.5372 33.6092 34.0717

Product 2.2212 -23.5230 -23.5524 -27.9119

Flash (300x250) Price Offer 3.4620 1.0540 1.9874 4.9752

Product 3.2155 -26.7275 -26.6279 -30.7119

Flash (728x90) Price Offer 4.5548 5.3249 5.1753 8.6858

GIF (160x600) Price Offer 0.0017 115.2477 107.3873 80.6858

GIF (300x250) Price Offer 0.0024 279.9628 273.8280 245.0363

GIF (728x90) Price Offer 0.0024 201.8751 197.3796 175.6671

Targeted Segments

Retargeting ------ 3.17005 21.3872 20.8412 17.5672

Male ------ 1.59892 -32.5273 -33.1476 -32.1784

Female ------ 3.12948 5.2674 5.0714 3.6905

Age ------ 11.31338 -2.8541 -2.5567 -1.4025

Message Content
Product 7.8150 -18.6164 -18.5501 -22.0608
Price 11.3920 12.7806 12.7266 15.1603

%change clicks: (t-1|t)(1) = 16.5601; (t-1|t)(2) = 16.8095; (t-1|t)(4) = 22.2620;

11
As a robustness check, we re-solved the one period (t-1|t)(1) problem over 1000 random draws from the
posterior, using a shorter period (T=80) given the computational complexity of solving P1. Table T2
reports the actual impressions and the mean (%) changes in impressions. Figure U1 shows the
distribution of the resulting % gains in total clicks.

Table T2: ALLOCATIONS: 1000 Random Posterior Draws: Model Re-Allocation Changes (%)
Actual T=80
Digital Format Exposure Impressions %(t-1|t)(1)

Product 1.0419 -2.9488

Flash (160x600) Price Offer 2.2292 14.3033

Product 1.0958 -13.3422

Flash (300x250) Price Offer 2.3132 0.7904

Product 1.5176 -16.7854

Flash (728x90) Price Offer 2.9951 2.8045

GIF (160x600) Price Offer 0.0014 141.7609

GIF (300x250) Price Offer 0.0013 353.5012

GIF (728x90) Price Offer 0.0015 263.3564

Targeted Segments

Retargeting ------ 1.7709 21.8426

Male ------ 1.1188 -26.2751

Female ------ 1.0855 1.0055

Age ------ 7.2218 -1.437

Message Content
Product 3.6553 -11.8092
Price 7.5417 5.7237

%change clicks: (t-1|t)(1) = 30.3221% , 95%HPDI= (7.1724, 62.6868)

12
Figure U1: Distribution of the %changes in clicks
Reallocation: %Clicks - 1000 Draws - T=80; (t-1|t)(1)
140

120

100

80

60

40

20

0
-20 0 20 40 60 80 100 120 140 160 180
%Change Clicks

13