Tales of Two Channels: Digital Advertising Performance Between AI Recommendation

and User Subscription Channels

Beibei Dong
Associate Professor of Marketing
College of Business
Lehigh University
bdong@lehigh.edu

Mengzhou Zhuang
Assistant Professor of Marketing
Faculty of Business and Economics
University of Hong Kong
mzhuang@hku.hk

Eric (Er) Fang

Professor of Marketing
Iacocca Chair of Business
Director of Center of Digital Marketing Strategy and Analytics
College of Business
Lehigh University
erf219@lehigh.edu

Minxue Huang (Corresponding author)

Professor of Marketing
Economics and Management School
Wuhan University
huangminxue@whu.edu.cn

1
 TABLE OF CONTENTS

WEB APPENDIX A. SUPPLEMENTARY TABLES AND FIGURES 3

WEB APPENDIX B. EXPERIMENT MATERIALS OF STUDY 2 9

WEB APPENDIX C. MATERIALS OF STUDY 3 AND STUDY 4 11

WEB APPENDIX D. MATERIALS OF STUDY 5 15

These materials have been supplied by the authors to aid in the understanding of their
paper. The AMA is sharing these materials at the request of the authors.

2
 WEB APPENDIX A. SUPPLEMENTARY TABLES AND FIGURES

Figure WA1. Exemplifiers of Subscription-based and Recommendation-based In-feed Channels

3
Figure WA2. Experiment Procedure

4
 Table WA1. Key Literature in the Relevant Research Streams
Article IV DV Method Key Findings
Native Ads
Wojdynski & Evans Disclosure format Ad evaluation Lab Middle or bottom disclosure position and disclosure words increase negative evaluations
(2016) and position experiments toward native ads, mediated by ad recognition.
Campbell & Evans Disclosure Ad attitudes Lab The companion of banner boosts ad recognition and leads to negative ad attitude;
(2018) companion banner experiments sponsorship transparency reduces such negative effects.
Sahni and Nair Disclosure type CTR, CR Field More prominent disclosures on search listings generate higher CTRs, and CRs come
(2020) experiment from clicking on corresponding organic listings.
Aribarg and Native vs. display CTR, attitude Field Native ad generates higher CTR but display ad creates greater visual attention, brand
Schwartz (2019) ad experiment recognition and platform trustworthiness.
Wang et al. (2019a) Serial position CTR, CR Field As serial position of native ads lowers, it affects CTR modestly but conversion
experiment drastically.
Ad Design
Chandy et al. (2001) Appeal type Referrals Secondary Emotional ads are more effective in mature markets and argument-based appeals more
data effective in newer markets.
MacInnis et al. Advertising Sales Secondary For TV commercials, emotional content is more likely to increase sales than rational
(2002) executional cues data appeal.
Bass et al. (2007) Advertising theme Wearout Secondary Informational ads (i.e., price and product) wear out faster than emotional ads (i.e., call
data stimulation, reconnect, and competition).
Goldfarb and Ad content and Purchase Field Matching ads to website content and increasing ad’s obtrusiveness independently
Tucker (2011) obtrusiveness intention experiment increase purchase intent; but in combination, do worse.
Teixeira et al. Emotion-induced Attention Lab Emotion-induced ads (joy and surprise) increase attention concentration and viewer
(2012) cues concentration, experiment retention. Surprise has a stronger effect on attention concentration and joy has a
viewer retention stronger effect on viewer retention.
Bruce et al. (2017) Ad image and flash CTR Secondary Animated ads (flash) increase online engagement than static ads (pictures); static
data formats can be effective for price ads and retargeting.
Chandrasekaran et Ad content Online search Secondary Increases in online search are driven by informational content but not by emotional
al. (2018) data content.
Guitart and Ad appeal Online search, Secondary Emotional ad content increases both online search and sales, while informational content
Stremersch (2020) sales data increases sales only; for sales, informational content works better for low-price products
while emotional appeal matches high-price products.

5
 Table WA2. Definition and Summary Statistics of Key Variables in Study 1
Definition Mean SD Min Median Max
Ad-level
Impression Number of daily ad exposures 25454.66 162038.14 1.00 674.00 5981854.00
Click Number of daily click-through 855.09 8544.36 0.00 12.00 622798.00
Conv Number of daily app downloads 7.98 55.20 0.00 0.00 2541.00
Recommend The ad served in the recommendation context 0.39 0.49 0.00 0.00 1.00
Direct The ad with direct download links 0.76 0.43 0.00 1.00 1.00
Information The ad with informational-oriented design 0.52 0.50 0.00 1.00 1.00
CumImpression The cumulative number of impressions 1237522.10 4499397.90 1.00 99243.50 66790592.00
DaySinceLaunch Number of days the ads have served 30.32 22.83 1.00 25.00 90.00
WordNum Number of words in ad text 15.45 2.31 11.00 16.00 21.00
Words/Sentence Number of words per sentence in ad text 9.14 4.49 4.00 7.50 21.00
Positive The positive attitude of the ad text, measured by paddlehub 0.05 0.07 0.00 0.00 0.45
Negative The negative attitude of the ad text measured by paddlehub 0.01 0.03 0.00 0.00 0.20
CharacterGender The gender of image characters, female=0, male=1, other=2 1.16 0.85 0.00 1.00 3.00
Human Ad image with human characters 0.95 0.44 0.00 1.00 2.00
Multiple Ad image with multiple characters 1.28 0.47 0.00 1.00 2.00
Interactions Ad image with character interactions 0.12 0.32 0.00 0.00 1.00
Consumer-level
Membership Weibo membership level, 0 is the lowest 0.40 0.84 0.00 0.00 4.00
Verification Whether the consumer has verified his/her true identity 0.50 0.50 0.00 0.00 1.00
Tenure Number of weeks since registered as member 169.52 61.17 0.00 163.14 651.14
Gender Gender of the Weibo user 0.39 0.49 0.00 0.00 1.00
Subscribers Number of users the consumer followed 1612.73 1362.91 0.00 1376.00 54847.00
Followers Number of users followed the consumers 45.93 106.36 0.00 26.00 28724.00
Posts Number of posts the consumer posted 27.45 41.33 0.00 19.00 5055.00

6
 Table WA3. Randomization Check
Panel A: Ad-level Covariates
Channel Link
Variables Recommend Subscribe Diff Indirect Direct Diff
Recommend .498(.501) .531(.502) -.034(.062)
Direct .692(.463) .662(.475) .030(.054)
Information .562(.498) .550(.499) .012(.058) .527(.500) .615(.489) -.087(.061)
Words 15.199(2.409) 15.185(2.429) .013(.281) 15.090(2.112) 15.406(2.951) -.317(.336)
Words/Sentence 9.240(4.680) 8.985(4.293) .255(.522) 8.731(3.898) 9.903(5.445) -1.171(.920)
Positive .040(.058) .052(.071) -.012(.008) .048(.066) .041(.063) .007(.008)
Negative .008(.027) .014(.033) -.007(.006) .011(.029) .011(.033) -.001(.004)
Character Gender 1.178(.945) 1.199(.841) -.021(.104) 1.164(.899) 1.240(.880) -.075(.110)
Human 1.014(.498) .927(.367) .087(.051) .980(.458) .948(.394) .032(.052)
Multiple 1.267(.474) 1.318(.508) -.051(.057) 1.303(.492) 1.271(.492) .033(.061)
Interactions .116(.322) .172(.379) -.056(.041) .129(.336) .177(.384) -.048(.046)
Number of ads 146 151 96 201
Panel B: Consumer-level Covariates
Channel Link
Variables Recommend Subscribe Diff Indirect Direct Diff
Membership Level .399(.836) .400(.836) .000(.000) .400(.836) .400(.838) -.001(.001)
Verified .500(.500) .500(.500) .000(.000) .500(.500) .500(.500) .000(.001)
Tenure 169.510(61.168) 169.520(61.166) -.010(.012) 169.518(61.166) 169.581(61.234) -.063(.069)
Gender .387(.487) .387(.487) .000(.000) .387(.487) .386(.487) .000(.001)
Subscriber (×100) 16.127(13.624) 16.128(13.633) -.001(.003) 16.128(13.631) 16.118(13.583) .010(.015)
Follower 45.908(106.700) 45.929(106.089) -.021(.020) 45.925(106.233) 45.885(104.110) .040(.118)
Post 27.451(41.425) 27.454(41.328) -.003(.008) 27.453(41.350) 27.463(41.107) -.009(.047)
Number of exposures 35,527,893 127,220,726 784,410 161,964,209
* p < .10, ** p < .05, *** p < .01.
Notes . In Panel B, we use only the first week of observations, to avoid the influence of ad selection. Standard deviations for means
and standard errors for differences are in parentheses.

7
 Table WA4. Summary Statistics of Variables Within and After the First Week
Mean SD Min Median Max
In the First Week
Impression 78282.16 346472.73 1.00 1537.00 5981854.00
Click 2484.15 18507.23 .00 26.00 622798.00
Conv 23.63 120.67 .00 .00 2541.00
Recommend .51 .50 .00 1.00 1.00
Direct .68 .47 .00 1.00 1.00
Information .56 .50 .00 1.00 1.00
After the First Week
Impression 14294.38 75756.21 1.00 576.00 2196754.00
Click 510.94 3926.74 .00 11.00 167460.00
Conv 4.67 23.51 .00 .00 588.00
Recommend .36 .48 .00 .00 1.00
Direct .78 .42 .00 1.00 1.00
Information .51 .50 .00 1.00 1.00

8
 Web Appendix B. Experiment Materials of Study 2

Table WB1. Summary Statistics of Variables in Study 2

Subscription Condition, Sample Size = 66
Mean SD Min Max
Enagement with organic content 5.27 1.10 2.50 7.00
Source credibility 4.65 1.23 1.00 7.00
Content control 5.09 1.32 1.00 7.00
Presentation fluency 5.23 1.19 2.00 7.00
Age 2.20 .61 2.00 5.00
Gender .53 .50 .00 1.00
Experience 3.03 .80 1.00 5.00
TikTok .77 .42 .00 1.00
Recommendation Condition, Sample Size = 66
Mean SD Min Max
Enagement with organic content 4.85 1.25 1.00 7.00
Source credibility 4.05 1.17 1.00 6.50
Content control 4.50 1.43 2.00 7.00
Presentation fluency 5.05 1.13 1.50 7.00
Age 2.11 .31 2.00 3.00
Gender .47 .50 .00 1.00
Experience 3.11 .93 1.00 5.00
TikTok .76 .43 .00 1.00

9
 Table WB2. Measurement Scale for Study 2
Variables Measurement Alpha/r
Engagement with Recall the organic content you just saw on the platform .82
organic content (1 = Strongly disagree, 7 = strongly agree)
1. I was greatly engaged in the organic content.
2. I paid a great deal of attention to the organic content.
Source credibility What do you think about the content creators of organic content you .87
just saw? (1 = Strongly disagree, 7 = strongly agree)
1. They are trustworthy
2. Their information is credible
Content control I choose which content creators to see. NA
(1 = Strongly disagree, 7 = strongly agree)
Presentation fluency Overall, what do you think about the transition/flow of organic .85
content? (1 = Strongly disagree, 7 = strongly agree)
1. The transitions between the content are smooth.
2. The pictures/videos flow naturally one after another.
Age Please select your age range NA
(1 = Under 18, 5 = 30 or above)
Gender Please indicate your gender NA
(0= Male, 1= Female, 2= Non-binary/third gender)
Experience How many hours do you typically spend on the platform? NA
(1 = less than 10 mins, 5 = more than 3 hours)

10
 WEB APPENDIX C. MATERIALS OF STUDY 3 AND STUDY 4

The participants were first directed to watch a 90-second video explaining the two feeding
channels (subscription and recommendation) (https://youtu.be/fsB-icv2t6M). After the video,
they participated in a short quiz which tested their understanding of the video content. The quiz
questions vary according to their assigned conditions as the first part of the manipulation.
Subscription:
1. According to the video, which tab in Google news is based on user subscription?
2. In which tab, you get to see posts from the sources you have subscribed to?
3. Which newsfeed tab feeds posts to you based on your own choice?
Recommendation:
1. According to the video, which tab in Google news is based on AI recommendation?
2. In which tab, you get to see news from sources that you might NOT have subscribed to?
3. Which newsfeed tab pushes news to you based on what the app recommends?

The correct answers were given to the participants if their answers were incorrect. Next,
participants were given a scenario where they were browsing a hypothetical social network app,
featuring either a subscription channel (“following” tab) or a recommendation channel (“for you”
tab) as the second part of the manipulation. The scenario descriptions are shown below:
Subscription
You are browsing a list of posts from the "Following" tab. Among all the posts, the
organic ones are from the sources you followed or subscribed to (recall the "User
Subscription" model discussed in the video). In other words, the organic posts are from
the sources of your own choices (followed or subscribed to).

Recommendation
You are browsing a list of posts from the "For You" tab. Among all the posts, the organic
ones are recommended by the app (recall the "AI Recommendation" model discussed in
the video). In other words, the organic posts do not come from the sources of your own
choices but are what the algorithm thinks you might like.

Then participants in both conditions were shown a screenshot that displayed five posts vertically,
including four organic posts and one in-feed ad about a vacation package (marked as sponsored,
in the 4th place). See Figure WC1 for details. The screenshot was identical between the two
conditions, except that on the top of the screen, it highlights the tab as either “following” or “for
you,” as the third part of the manipulation. We then asked participants to rate their intention to
click on the 4th post to learn more and the extent to which they found the post intrusive on 7-
point scales. To minimize demand artifacts, we did not explicitly disclose that the post was an
ad.

11
Figure WC1. Screen Shots Used in the Scenarios in Studies 3 and 4
Subscription Recommendation

Notes. The fourth post is the in-feed ad (sponsored post)

12
 Table WC1. Summary Statistics of Variables in Studies 3 and 4
Study 3 Subscription condition Recommendation condition
Sample size = 201 Sample size = 200
Variables Mean SD Min Max Mean SD Min Max
Click intention 2.97 1.94 1.00 7.00 3.30 2.08 1.00 7.00
Ad intrusiveness 3.44 1.97 1.00 7.00 3.00 1.67 1.00 7.00
Age 3.97 1.20 2.00 7.00 4.00 1.19 2.00 8.00
Gender 1.52 .53 .00 2.00 1.54 .53 .00 2.00
Experience 4.24 1.15 1.00 6.00 4.14 1.13 1.00 6.00
Income 5.86 3.04 1.00 12.00 5.84 3.25 1.00 12.00
Study 4 Subscription condition Recommendation condition
Sample size = 147 Sample size = 153
Variables Mean SD Min Max Mean SD Min Max
Ad Recognition 5.95 1.23 2.00 7.00 5.95 1.07 2.00 7.00
Age 4.35 1.21 2.00 8.00 4.25 1.26 2.00 7.00
Gender 1.50 .50 1.00 2.00 1.42 .53 .00 2.00
Experience 4.10 1.35 1.00 6.00 3.99 1.27 1.00 6.00
Income 5.97 2.92 1.00 12.00 6.20 3.05 1.00 12.00

13
 Table WC2. Measurement Scale for Studies 3 and 4
Variables Measurement Alpha/r
Click intention 1.When browsing the social network, how likely will you click this post to learn .98
(Study 3) more about the travel package?
2.When browsing the social network, to what extent will you click this post to
learn more information about the travel package?
(1 = Strongly disagree, 7 = strongly agree)
Ad intrusiveness To what extent do you consider that this post was (1 = Strongly disagree, 7 = .89
(Study 3) strongly agree)
1. Disturbing
2. Interfering
3. Obtrusive
Ad Recognition To what extent do you think there is sponsored advertisement among the NA
(Study 4) newsfeed you were browsing? (1 = Definitely not, 7 = Defintely Yes)
Age Please select your age range NA
(Studies 3 & 4) (1 = Under 18, 9 = 85 or older)
Gender Please indicate your gender NA
(Studies 3 & 4) (0 = Prefer not to say, 1 = Male, 2 = Female)
Experience How much time do you typically spend on these social network apps daily? NA
(Studies 3 & 4) (1 = None, 6 = more than 3 hours)
Income May I know your annual income? NA
(Studies 3 & 4) (1 = Less than $10,000, 12 = More than $150,000)

14
 WEB APPENDIX D. MATERIALS OF STUDY 5

Table WD1. Mean Comparisons between Recommendation and Subscription Conditions

Recommenation Subscription Difference
Organic Post Gaze time 2370.662 3559.144 -1188.483**
(323.157) (424.066) (536.070)
Gaze count 13.196 19.556 -6.359***
(1.597) (1.713) (2.346)
Ad Gaze time 2232.462 1030.148 1202.313**
(559.970) (151.506) (570.596)
Gaze count 11.423 5.889 5.534**
(2.225) (.602) (2.267)
One post after vs. one post Gaze time -449.885 -1600.481 1150.597*
before ad (487.699) (422.677) (643.841)
Gaze count -2.923 -7.667 4.744*
(1.959) (2.023) (2.819)
Two posts after vs. two posts Gaze time -564.981 -1915.148 1350.167*
before ad (593.150) (451.600) (741.945)
Gaze count 4.058 9.852 -5.794
(2.571) (2.324) (3.461)
Three posts after vs. three Gaze time -263.744 -896.593 632.849
posts before ad (366.942) (307.675) (477.429)
Gaze count -2.449 -4.272 1.823
(1.548) (1.601) (2.229)
Four posts after vs. four posts Gaze time -170.933 -1122.231 951.299**
before ad (318.551) (338.255) (465.345)
Gaze count -2.183 -5.343 3.160
(1.273) (1.891) (2.297)
All posts after vs. all posts Gaze time -633.737 -1588.401 954.664**
before ad (223.340) (342.106) (411.853)
Gaze count -4.561 -8.025 3.464
(1.105) (1.892) (2.212)
* p < .10, ** p < .05, *** p < .01.
Notes . Gaze time is in millisecond; standard errors are reported in the parentheses.

15
 Figure WD1. Experiment Scenarios
Subscription Recommendation

Notes. The two conditions are different in three aspects (highlighted in red rectangles). In the
recommendation (subscription) channel, 1) the posts are displayed under the “For you”
(“Following”) tab and 2) a “follow” button is (not) displayed at the right corner of the post.

16
 Figure WD2. Heatmap for Study 5
Subscription condition Recommendation condition
1st
position
Organic

2nd
position
Organic

17
 Figure WD2. Heatmap for Study 5 (cont’)
Subscription condition Recommendation condition
3rd
position
Organic

4th
position
Organic

18
 Figure WD2. Heatmap for Study 5 (cont’)
Subscription condition Recommendation condition
5th position
Ad

6th position
Organic

19
 Figure WD2. Heatmap for Study 5 (cont’)
Subscription condition Recommendation condition
7th
position
Organic

8th
position
Organic

20
 Figure WD2. Heatmap for Study 5 (cont’)
Subscription condition Recommendation condition
9th position
Organic

10th
position
Organic

21