1

2
Am I Playing Better Now? A Study of Effects of Adaptive 58
59
3 Synchronization in 60Hz Gameplay 60
4 61
5 62
6 ANONYMOUS AUTHOR(S) 63
7 64
Adaptive synchronization (ASync) technology matches formerly regular in significance. we report here on two new studies of the effects of
8 display refreshes to irregular frame updates, improving frame rates, and 65
ASync on player performance and experience at 60Hz frame rates.
9 interactive latency. In a previous study of gaming at the 30Hz frame rates 66
10 common on consoles, players of Battlefield 4 were unable to discern when 67
1.1 Contributions
11 ASync was in use, but scored higher with ASync and were affected emotion- 68
12 ally. We build on that study by examining ASync’s effects at the 60Hz frame With these two experiments: 69
13 rate more common in PC gaming, with an implicit experimental measure, 70
• We describe the first examination of ASync’s effects on game-
14
and over a longer period of time. We find that ASync is still imperceptible 71
at 60Hz and that its effects are more focused than at 30Hz. Nevertheless, play at the 60Hz frame rates common on PCs.
15 72
those effects persist. In particular, ASync can improve experience, and also • We introduce a new application of the implicit association
16 73
performance for veteran players who gain familiarity with their game. We test (IAT) [21, 29] to measure gaming engagement of players
17 74
also find suggestive differences in the experience of novice and veteran game with and without G-SYNC.
18 75
players. • We learn that while ASync’s effects at 60Hz are more limited
19 76
than at 30Hz, it can continue to improve gaming performance
20
CCS Concepts: • Human-centered computing → Displays and imagers; 77
Empirical studies in HCI ; • Applied computing → Computer games. and experience.
21 78
• We find differences in the experience of novice and veteran
22 Additional Key Words and Phrases: refresh rate, frame rate, latency, computer 79
gamers that may suggest fundamental differences in how
23 games, user experience 80
they evaluate gameplay.
24 81
25
ACM Reference Format: 82
Anonymous Author(s). 2018. Am I Playing Better Now? A Study of Effects 2 ADAPTIVE SYNCHRONIZATION
26 83
of Adaptive Synchronization in 60Hz Gameplay. In Woodstock ’18: ACM For decades, fixed display refresh rates were higher than rendered
27 84
Symposium on Neural Gaze Detection, June 03–05, 2018, Woodstock, NY . ACM, frame rates. But as GPUs improved and frame rates overtook refresh
28 85
New York, NY, USA, 12 pages. https://doi.org/10.1145/1122445.1122456
29
rates, GPUs more often had to wait for the display to refresh to avoid 86
30
image tearing, increasing latency. ASync eliminates such waits by 87
1 INTRODUCTION synchronizing refreshes to frame updates. This eliminates not only
31 88
32 Latency affects video gameplay [13, 61], which can be quite quickly delays caused when GPUs are faster than displays, but also those 89
33 paced. Because fixed display refresh rates began to limit latency caused when GPUs are slower. In this case, the GPU misses a dis- 90
34 improvements, graphics hardware (GPU) and collaborating display play refresh, and the display repeats the last frame. This introduces 91
35 manufacturers introduced adaptive synchronization (ASync) tech- “jitter”, highly variable frame rates particularly annoying to gamers. 92
36 nology, which allows GPUs to avoid waits when delivering imagery NVIDIA calls its proprietary ASync technology G-SYNC, AMD’s 93
37 to displays. NVIDIA claims that the tear-free immersion of its G- more open alternative is called FreeSync. 94
38 SYNC technology will beat gamers’ expectations [54]. Similarly, Studying the effects of ASync on user experience is a novel line 95
39 AMD promises to “puts an end to choppy gameplay” with FreeSync of research. Poth et al. [55] used G-SYNC to maximize temporal 96
40 [2]. resolution by setting GPU frame rate just above display refresh rate. 97
41 Watson et al. [61] recently showed that ASync can improve gam- Though the temporal improvement was only several milliseconds, 98
42 ing performance and experience, in the 30Hz frame rate gameplay viewers recognized letters better as a result, demonstrating ASync’s 99
43 typical of many gaming consoles. However, PC players often have value outside of gaming. Within gaming, Watson et al. [61] studied 100
44 higher frame rates on their systems [40] and prefer them if possible. the effects of ASync on players performance and experience in a 101
45 Gameplay performance at higher frame rates is also important in 30Hz gaming environment. Players were unable to identify gaming 102
46 esports [10, 33], which is already a billion-dollar industry [52] with sessions using ASync, even though their game scores improved 103
47 hundreds of millions of viewing hours [53], and continues to grow with it, and their emotions were affected by it. As measured by both 104
48
scores and surveys of emotion, ASync interacted with playing time, 105
49
Permission to make digital or hard copies of all or part of this work for personal or game content and the general gaming expertise of players. 106
classroom use is granted without fee provided that copies are not made or distributed
50 for profit or commercial advantage and that copies bear this notice and the full citation 107
51 on the first page. Copyrights for components of this work owned by others than ACM 3 LATENCY’S EFFECTS ON GAMING 108
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
52
to post on servers or to redistribute to lists, requires prior specific permission and/or a The effects of temporal sampling and delay on human performance 109
53 fee. Request permissions from permissions@acm.org. and experience have been the focus of decades of research, the bulk 110
54 Woodstock ’18, June 03–05, 2018, Woodstock, NY of it concerned with applications other than gaming [11, 12]. In our 111
© 2018 Association for Computing Machinery.
55
ACM ISBN 978-1-4503-9999-9/18/06. . . $15.00 discussion of related work, we concentrate first on latency’s impacts 112
56 https://doi.org/10.1145/1122445.1122456 on performance, then on how it affects experience. In both cases, 113
57 114

1
 Woodstock ’18, June 03–05, 2018, Woodstock, NY Anon.

115 gaming-related latency research is limited, so we briefly broaden Table 1. Participants vs. Expertise and ordering 172
116 our discussion. in the first experiment 173
117 174
118
3.1 Effects on Performance Gaming Expertise 175
119 <5 >5 total 176
The impact of latency on gameplay performance depends on the
120 177
nature of the game’s tasks. Claypool et al. [14] studied the effect G-SYNC first 9 8 17
121 178
of delay on selecting a moving target with a mouse in a custom G-SYNC second 7 8 15
122 179
game called “Juke”. Delay’s impact depended on the target’s angle total 16 16 32
123 180
and frequency of motion. Dick et al. [20] analyzed the effects of net-
124 181
work latency, jitter, and player skill on perception in four different
125 182
multiplayer games, and learned that effects depended heavily on Frommel et al. [25] used dynamic difficulty adjustment (DDA) to
126 183
the game being played. In their two studies, Long et al. [47] varied respond to players self-reports of boredom and frustration, cam-
127 184
latency while measuring player performance and experience, and ouflaged in dialog with non-player characters (NPCs), and found
128 185
found that latency’s effects depended in particular on game speed, that it was more effective than simpler DDA control schemes. Oth-
129 186
with faster-paced games affected by smaller latencies. In an esports ers suggest an alternative to self-reports, using machine learning
130 187
environment, Spjut et al. [58] found that latency affected perfor- processing elements of game performance and input to predict emo-
131 188
mance on some tasks more than others, while few were affected by tion [26, 56]. NPCs themselves also have a strong relationship to
132 189
frame rate (with constant latency). experience [7, 23, 35, 36, 43, 44]. It can be tempting to conclude that
133 190
Although the reported thresholds at which latency has effects experiencing positive emotion implies a good gaming experience,
134 191
vary widely (23 to 500ms), play in real-time games such as first- but a range of research [6, 8, 16, 28] suggests that this relationship
135 192
person shooters (FPSs) is susceptible to even small amounts of lag. is more complex, with negative emotions often creating a richer,
136 193
FPS games are fast-paced, with accurate aim and quick reaction better experience.
137 194
crucial for good performance. FPS players are aware of latency,
138 195
with studies showing that 100ms latency is noticeable, and 200ms is 4 EXPERIMENT 1: G-SYNC WITHIN PARTICIPANTS
139 196
annoying [5]. Others found that methods compensating for latency
140 Watson et al. [61] demonstrated that ASync can improve gaming 197
work well with FPS games [37, 45]. For example, Quake IV uses
141 performance and experience in systems with 30Hz frame rates, and 198
built-in algorithms to compensate for packet loss [62].
142 found intriguing relationships between ASync’s effects and the char- 199
Outside of gaming, even slight delays in touch interfaces (e.g.
143 acteristics of both games and gamers. Our goals were to investigate 200
trackpads, tablets and phones) can affect human performance. Janzen
144 these same questions in a more PC-centric and future-proof 60Hz 201
et al. [38] found that 30Hz frame rates made it difficult for users to
145 gaming context, and to study those relationships more deeply. To 202
click on moving targets via laptop touchpads. In a study by Jota et
146 do so, we carefully designed and evaluated a 60Hz PC gaming en- 203
al. [39], latencies as low as 25ms affected dragging on direct-touch
147 vironment, and introduced an implicit measure of experience, the 204
surfaces like tablets. Deber et al. [18] followed up this work, finding
148 Implicit Association Test (IAT) [30]. 205
that direct-touch devices (e.g., phones and tablets) are more sensitive
149 206
to latency than indirect-touch devices (e.g., mice and touchpads).
150 4.1 Participants 207
Long et al. [48] studied game input devices (mouse, touchscreen,
151 32 students participated in our experiment, all undergraduates 208
gamepad, and drawing tablet), and learned that latency affects each
152 or graduates in our Computer Science department. We recruited 209
device differently. Only the touchscreen was unaffected by latency
153 them using email announcements, and compensated them with 210
in targeting stationary objects. In motion-tracked systems such as
154 extra credit in their course or with $10 Amazon gift cards. Ta- 211
virtual reality, Ellis et al. [22] used machine learning to recognize
155 ble 1 shows how we assigned participants to different between- 212
poses more rapidly, reducing interactive latency.
156 participant groups in our experiment. 213
157 214
158 3.2 Effects on Experience 4.2 Design 215
159 A few research projects have found that low latency can improve We used a 3-factor mixed design (2GS × 2SN × 2GE). Figure 1 shows
216
160 game experience. Long et al. varied latency during gameplay, and our independent variables and dependent measures, and how they
217
161 found changes in enjoyment, competence, frustration, and perfor- related to experimental procedure.
218
162 mance [47]. Claypool et al. [15] varied game speed and delay while 219
163 players used a mouse to select a moving target. Delay harmed expe- 4.2.1 Independent Variables. To study the effect of ASync, all par- 220
164 rience more than performance. Hoelfeld et al. [34] found that the ticipants played a session in Battlefield 4 (B4) with and without 221
165 experience of novice Minecraft players was not affected by delay, G-SYNC (GS). Thus our G-SYNC independent variable was within 222
166 but speculated that expert players might be more affected. subjects. 223
167 Latency aside, research suggests that gaming experience has com- As participants play, their familiarity with the game grows, while 224
168 plex relationships to the characteristics of gamers, and the games the game becomes more challenging. We investigated these effects 225
169 they are playing [1, 46]. In particular, the relationship between game by dividing gameplay into two sessions (SN ). Our session variable 226
170 difficulty and experience has been studied extensively. For example, was also within subjects, although it was not completely crossed 227
171 228

2
 Am I Playing Better Now? A Study of Effects of Adaptive Synchronization in 60Hz Gameplay Woodstock ’18, June 03–05, 2018, Woodstock, NY

229 286
230 287
231 288
232 289
233 290
234 291
235 292
236 293
237 294
238 295
239 296
240 297
241 298
242 299
243 300
244 301
245 302
246 Fig. 1. Procedure for our first G-SYNC within participants experiment, and how it relates to independent variables and dependent measures. Boldface indicates 303
247 independent variables, while blue blocks represent gameplay with G-SYNC on, red blocks with G-SYNC off. 304
248 305
249 306
250 with G-SYNC: participants did not play both with and without G- player mission, B4 awards points for each kill: regular 100 points, 307
251 SYNC in each session. headshot 125, multi- 100 points each, adrenaline 150 (vs. a low 308
252 To examine the impact of general gaming expertise, participants health enemy), melee 125, streak 150, and squad 50. We considered 309
253 told us whether they had played games for less than 5 years, or alternative measures of performance that might be less sensitive 310
254 more. In Watson et al.’s work [61], players with 5 years of gameplay to variation in within-game difficulty and easier to generalize to 311
255 often performed better and experienced differently than players other games, but did not find any that responded differently to 312
256 with less experience. We also considered a measure of expertise our independent variables, or were clearly more generalizable. We 313
257 with more recency, hours played per week (more or less than 6). examine this issue in more detail in Section 6.1. 314
258 However, this measure was quite correlated with years of gaming Subjective Duration Assessment (SDA). SDA is an implicit measure 315
259 (<5 years: 69% < 6 hours; >5: 75% > 6). In our text, we refer to those of subjective difficulty introduced by Czerwinski et al. [17], who 316
260 who played games less than 5 years as “novice gamers”, and to those argued that users consistently underestimate actual task time when 317
261 with more as “veteran gamers.” Our gaming expertise variable (GE) the task feels easy, and overestimate task time when it feels difficult. 318
262 was between participants. Bederson also argues that SDA could be used as a measure of flow 319
263 [4], the pleasing state of full involvement in an activity, which is 320
264 4.2.2 Dependent Measures. We will discuss each of our dependent closely related to difficulty [51]. Participants estimated the number 321
265 measures in the order participants encountered them, as shown on of minutes they had played after each gaming session. We then 322
266 our procedural diagram in Figure 1. subtracted the actual number of minutes they had played from their 323
267 Self-Assessment Manikin (SAM). SAM is a simple, pictorial survey estimate to define that session’s SDA measure. 324
268 for assessing a person's emotional state [9], based on the three- Display Identification (DI). To measure the perceptibility of ASync, 325
269 dimensional model described by Mehrabian and Russell [50]. In after their last gaming session we asked participants, “Which moni- 326
270 that model, the pleasure dimension (SAM-P) places current emotion tor do you think was equipped with G-SYNC, the improved display?” 327
271 on an axis labeled with semantic differential adjective pairs such They replied by identifying the gaming session with the display 328
272 as “unhappy-happy,” “annoyed-pleased,” and “unsatisfied-satisfied.” they found better. The DI measure is the ratio of participants who 329
273 Arousal (SAM-A) is labeled with pairs such as “relaxed-stimulated,” correctly identified the G-SYNC session. 330
274 “calm-excited,” and “sluggish-frenzied.” Finally, dominance (SAM-D) Implicit Association Test (IAT). As an implicit measure of game- 331
275 corresponds to “controlled-controlling,” “influenced-influential,” and play engagement, we used the IAT. Implicit measures do not rely 332
276 “cared for-in control.” Participants described their emotions using primarily on conscious thought in participants, and are useful when 333
277 SAM three times: before, between and after gameplay sessions. We researchers are concerned about possible cognitive biases. One ex- 334
278 used the difference between succeeding SAM assessments to assess ample is social desirability bias, which drives participants to give 335
279 the emotional change after gameplay sessions, which resulted in a the answers they think researchers want. Since its introduction in 336
280 between - before SAM difference and an after - between difference. 1998 [30], the IAT has been widely adopted, improved and validated 337
281 Game Performance (GP). To measure player performance in B4, we [31, 32, 42]. The IAT measures subconscious association between 338
282 used only its own single-player scoring system. After each gaming two cognitive constructs, often with one “attribute” construct char- 339
283 session, we recorded the number of additional points earned in that acterizing a “target” construct. For example, the IAT has been widely 340
284 session (points at end of session less points at beginning). In a single 341
285 342

3
 Woodstock ’18, June 03–05, 2018, Woodstock, NY Anon.

343 400
344 401
345 402
346 403
347 404
348 405
349 406
350 407
351 408
352 409
353 410
354 411
355 412
356 413
Fig. 2. The five-stage Implicit Association Test procedure, adapted from Fig. 3. An example screen during the IAT’s block 3. Participants press “E” if
357 they classify the center example with the left side of the top construct, and 414
[21]. In discrimination blocks 1 and 2, participants classify examples of the
358 attribute (engagement) and target (gaming session) constructs. In combined “I” if they classify with the right. 415
359 block 3, they classify examples of both constructs at once. In discrimination 416
360 block 4, the direction of the attribute construct is reversed, and participants 417
361 classify its examples. Finally in combined block 5, participants again classify 418
both constructs, with the attribute construct reversed. maximum frame rate of 62Hz and “high” level of detail (B4’s second-
362 419
highest setting). This resulted in a mean frame rate of 57.683 Hz (σ
363 420
= 2.22) without G-SYNC, and a mean frame rate of 59.467 Hz (σ =
364 421
1.83) with G-SYNC. When G-SYNC was off, VSync was on. Players
365 422
used to measure associations between the socially sensitive target used mouse and keyboard to interact with B4.
366 423
construct of race, and the attribute construct of goodness. To de- We implemented the IAT using Meade’s FreeIAT software (see
367 424
termine associative strength, the IAT asks participants to classify Figure 2 for an overview) [49], which uses the updated IAT proce-
368 425
examples (phrases or imagery) of both constructs, while they view dure described in [31]. As examples of the gaming session target
369 426
a directional pairing of those constructs (Figure 3). If participants construct, we used phrases identifying the portion of the B4 mis-
370 427
implicitly agree with the pairing, they will make the classification sion played, such as “protecting Irish,” “helicopter,” “safe house,”
371 428
more quickly. We used the IAT to measure association between etc. As examples of the engagement attribute construct, we used
372 429
engagement as an attribute construct and gaming session as a target phrases such as “I was absorbed,” “It was demanding,” “It was confus-
373 430
construct, after all gameplay was complete. ing” drawn from Wiebe et al.’s self-reported engagement survey for
374 431
gaming [63]. Participants input their classification of examples by
375 432
4.3 Apparatus tapping on E for the left-hand side of the construct pairing, or I for
376 433
the right-hand side (3). To determine associative strength, the IAT
377 The game Battlefield 4 is supported by NVIDIA drivers and is popular finds the difference between the average classification times in the
434
378 among FPS video gamers [27]. It is visually rich, and requires rapid two combined blocks. We ordered the phrases such that negative
435
379 response to feedback. It is also the game used by Watson et al. [61], differences indicated more engagement with the second session,
436
380 allowing us to better replicate and extend their work. and positive differences more engagement with the first session.
437
381 In choosing our experimental hardware configuration, our pri- 438
382 mary goal was to realize a frame rate in B4 of roughly 60Hz. This 4.4 Procedure and Task 439
383 frame rate is certainly not at the high end of today’s gaming “rigs,” 440
When participants arrived, we randomly determined whether they
384 but is twice the frame rate used by Watson et al. [61], is commonly 441
would begin gameplay with or without G-SYNC. They then began
385 cited by gamers as a desirable goal [57, 59], is a widespread refresh 442
moving through seven experimental stages (Figure 1), which took
386 rate setting in displays, and approaches the peak rate at which 443
them an average of 44.5 minutes to complete. We discuss these
387 display flicker can be perceived [41]. 444
stages below.
388 In making our configuration choice, we varied PC and GPU hard- 445
389 ware, display refresh rates, and B4 settings including maximum 4.4.1 Before Gameplay. At the beginning of the experiment, we 446
390 frame rate and level of detail. Our secondary goals were to achieve thanked participants for their participation and told them that they 447
391 a meaningful improvement by G-SYNC of frame time mean and could pause or halt the experiment at any time. We informed them 448
392 standard deviation, and to avoid any configurations very atypical for that the experiment studied the effect of NVIDIA’s G-SYNC in gam- 449
393 gamers. To measure frame rates, we used the Fraps tool [24], with- ing, adding that they would use G-SYNC in one of their gaming 450
394 out its video recording function. B4 is now six years old, so our final sessions, but they would not know which until the experiment’s 451
395 configuration is modest and used an Intel Core i7-870 2.93GHz CPU end. We explained that we would ask them to describe their emo- 452
396 with 8GB RAM operated by 64-bit Windows, an NVIDIA GeForce tions and thoughts about their gameplay and wanted them to be 453
397 GTX 1050 Ti GPU, an Acer XB240H-A G-SYNC monitor displaying honest in their responses, reminding them that there is no “wrong” 454
398 1920×1080 pixels and using a 144Hz refresh rate, with B4 set to a answer and their responses would be anonymous. Participants gave 455
399 456

4
 Am I Playing Better Now? A Study of Effects of Adaptive Synchronization in 60Hz Gameplay Woodstock ’18, June 03–05, 2018, Woodstock, NY

457 informed consent for their participation and we assured them that 514
458 their data would be protected. 515
459 We then surveyed participants to ask which gaming expertise 516
460 group they belonged to, and to assess their emotional state using 517
461 the SAM. This pre-gameplay SAM data allowed us to control for 518
462 individual differences in participant emotion that did not result from 519
463 gameplay. 520
464 To introduce the game and its interface to participants, we al- 521
465 lowed them a practice game session using B4’s “BAKU” mission 522
466 in the “easy” mode. Participants started at the beginning of the 523
467 mission and halted when they felt comfortable with the game, or 524
468 when they had been playing for 10 minutes. When they died in the 525
469 game, B4 would let them restart from the most recent checkpoint. 526
470 With experimenter guidance, participants learned how to use keys 527
471 and mouse to control movement and weapons, and familiarized 528
472 themselves with B4’s maps, weapons, and help system. Participants 529
Fig. 4. Mean B4 scores from our first G-SYNC within participants experi-
473 beginning gameplay with G-SYNC on practiced with G-SYNC on, 530
ment, grouped by session and gaming expertise. Error bars show standard
474 those beginning without practiced with G-SYNC off. error.
531
475 532
476
4.4.2 First Gaming Session and Measures. Participants then pro- 533
477
ceeded to the experiment’s first gaming session, continuing with 534
478
B4’s “BAKU” mission in the “easy” mode. The mission required play- off conditions was only about 3Hz (centered around 60Hz), which 535
479
ers to find teammates, acquire weapons, exit a school building, fight translates to a ~0.8ms difference in frame time. Such temporal fre- 536
480
their way through enemies, and finished when they reached an open quency differences are at perceptual limits [60], and these latency 537
481
field. Participants played with G-SYNC if we randomly assigned differences are well below reported limits (e.g. [3]). Moreover, in a 538
482
them to that group, without G-SYNC otherwise. They continued similar experiment with larger temporal differences, Watson et al. 539
483
until they completed the mission (µ = 9.5 minutes), or had been [61] found that G-SYNC’s effect was imperceptible. We therefore 540
484
playing for 15 minutes. We then recorded participants’ B4 score for hypothesized that G-SYNC would: 541
485
the GP measure, surveyed participants to obtain the SAM measure, 542
Be imperceptible. Participants would not be able to identify
486
and asked participants to estimate their playing time for the SDA 543
G-SYNC reliably.
487
measure (participants were unaware of any session time limit). We 544
488
then toggled G-SYNC for all participants. 4.5.2 Performance. Research consistently shows that improved 545
489 frame rates and latencies improve task performance [18, 39]. Watson 546
4.4.3 Second Gaming Session and Measures. The portion of the
490 et al. [61] also found that G-SYNC improved gaming performance. 547
mission participants played in this session was more difficult, con-
491 However, G-SYNC’s temporal effects in our experiment were less 548
taining a larger number of enemy soldiers and vehicles. Participants
492 than the temporal manipulations in those experiments. We hypoth- 549
began the mission in the same open field at which they finished the
493 esized that G-SYNC would: 550
first session, and then attempted to cross an open field under intense
494 enemy fire to reach a large drainage pipe. Play halted when they Improve performance slightly. Participants would score a few 551
495 reached the pipe, or 15 minutes had passed. In fact, all participants more points with G-SYNC than without it. 552
496 used the full 15 minutes for this session. We again recorded partici- 553
497
4.5.3 Experience. Research relating experience to delay is limited, 554
pants’ B4 scores for our GP measure, surveyed them for our SAM but suggests that experience can respond strongly to frame rate and
498 measure, and asked them for a playing time estimate for the SDA 555
499
latency, and that this response interacts with gaming practice, con- 556
measure. We also asked participants to identify the gaming session tent and expertise. Based on Watson et al.’s work [61] and G-SYNC’s
500 that used G-SYNC for our DI measure. Participants then performed 557
501
more modest temporal effects in our experiment, we expected that 558
the IAT procedure to give us an implicit measure of how engaged G-SYNC would:
502 they were in each gaming session. We then informed participants 559
503 which session used G-SYNC. Improve experience slightly. With G-SYNC, participants would 560
504 experience more pleasure and control in the SAM measure, 561
505 4.5 Hypotheses and slightly more engagement in the IAT measure. In the 562
506 SDA measure, any effects would be minor. Gaming expertise 563
We based our experimental expectations on prior research, and in
507 would moderate many of G-SYNC’s experiential effects. 564
particular on the results of Watson et al. [61].
508 565
509 4.5.1 Perceptibility. Our display identification (DI) measure indi- 4.6 Results 566
510 cates the perceptibility of G-SYNC, with accuracy near chance levels In this section, we report effects on perceptibility, performance, and 567
511 (50%) being evidence that G-SYNC is imperceptible. In our experi- experience, discussing significant (p < .05) and marginal (p < .1) 568
512 ment, the average difference between the G-SYNC on and G-SYNC main effects and interactions. To analyze most of our results, we 569
513 570

5
 Woodstock ’18, June 03–05, 2018, Woodstock, NY Anon.

571 calm
Table 2. ANOVA of the within participants Experiment 628
1.0
572 629
573 measure effects distribution p value η2 630
574 0.5 631
GP SN F(1,56) = 8.22 = 0.005 = 0.12
575 632
GE GP GE F(1,56) = 6.48 = 0.013 = 0.103
SAM-A

576 633
0.0 <5 GP SN × GE F(1,56) = 6.73 = 0.012 = 0.107
577 634
>5 SAM-A SN F(1,56) = 5.35 = 0.024 = 0.087
578 635
SAM-A SN × GE F(1,56) = 5.35 = 0.024 = 0.087
579 −0.5 636
580 637
581 638
582 excite −1.0 639
583 S1 S2 the same increases in excitement in the first and second sessions 640
584 (S1: µ = −0.563, SE = 0.223; S2: µ = −0.563, SE = 0.241; p = 1, d = 0). 641
Session G-SYNC’s only effect was on game engagement, as measured
585 642
586 by the IAT. Standard methodology reports association strength as 643
Fig. 5. Mean SAM-Arousal from our first G-SYNC within participants exper- differences in classification times using Cohen’s d effect size [31],
587 644
iment, grouped by session and gaming expertise. Error bars show standard with a value of 0.8 or larger called a “strong” association, 0.5 or
588 645
error. larger “moderate” and 0.2 or larger “slight”. Significance of associa-
589 646
590 tions are analyzed using one-sided T-tests (comparing d to 0), while 647
591 significant differences in association strength are analyzed using 648
592 two-sided T-tests (e.g. [19]). FreeIAT marked all of one participant’s 649
performed a 3-factor between-participants ANOVA with G-SYNC
593 times as too fast; we removed them from analysis. Overall, partici- 650
(GS), session (SN ) and gaming expertise (GE) as factors. This between-
594 pants found the second session more engaging, but this association 651
participants analysis simplifies comparisons with our second ex-
595 did not reach even slight strength (d = -0.149). However, among 652
periment. Moreover, because GS and SN were not fully crossed, we
596 those using GS in the second session, this association reached a 653
could not use them as within factors in a 3-factor mixed ANOVA.
597 slight but significant strength (d = -0.251, t(13)=-2.86, p=0.014), in- 654
We compared interaction means with contrasts.
598 dicating that GS helped participants feel slightly more engaged in 655
599 4.6.1 Perceptibility. Participants identified the G-SYNC display cor- the second session. Among those using GS in the first session, any 656
600 rectly 53% of the time. This was not significantly different from association of engagement with session was absent (d = -0.070). 657
601 chance (50%) (χ 2 (1,32) = 0.03, p = .85). Participants could not reli- Whether or not GS was used in the first or second session also had 658
602 ably discern sessions with G-SYNC from sessions without it. a marginally significant affect on association strength (t(26.62)=1.92, 659
603 p=0.066). 660
4.6.2 Performance. G-SYNC had a negligible and insignificant effect
604 661
on B4 scores. Table 2 shows significant effects. Gaming expertise had
605 4.7 Discussion 662
a significant effect, with the scores of veteran players 40% higher
606 We begin discussion of these results by examining our hypotheses 663
than those of novice players (<5: µ = 1978.1, standard error (SE)
607 about G-SYNC’s effects, and continue with an examination of session 664
= 231.1; >5: µ = 2774.3, SE = 238.8). Session also had a significant
608 and expertise’s impacts. 665
effect, with scores in the second session 45% higher than those in the
609 666
first (S1: µ = 1942.3, SE = 41.7; S2: µ = 2810.1, SE = 326.8). Moreover,
610
GE and SN interacted significantly (Figure 4): while the scores of 4.7.1 Hypotheses About G-SYNC’s Effects. We verified our percepti- 667
611
novice players did not improve in the second gaming session (S1: bility hypothesis: participants could not reliably identify the gaming 668
612
µ = 1953.1, SE = 48.0; S2: µ = 2003.1, SE = 467.3; p = .9975, d = session using G-SYNC. This is not surprising, considering that G- 669
613
0.038), the scores of veteran players did (S1: µ = 1931.4, SE = 69.7; SYNC was also imperceptible in the study by Watson et al. [61], and 670
614
S2: µ = 3617.1, SE = 369.0; p = .0015, d = 1.587). that G-SYNC in this experiment had less temporal impact at 60Hz 671
615 than in Watson et al.’s study at 30Hz. We could not confirm our per- 672
616 4.6.3 Experience. We found no significant effects on experienced formance hypothesis, since G-SYNC did not improve performance 673
617 difficulty or flow as measured by SDA, nor on reported levels of at all. However, we were correct in our expectation that G-SYNC’s 674
618 pleasure and control as measured by SAM valence and dominance. effect on performance would be less than it was in Watson et al., 675
619 But session did affect SAM arousal significantly (Table 2), with partic- again likely due to the reduced temporal impact of G-SYNC at 60Hz. 676
620 ipants gaining more excitement in the first session than the second Our experience hypothesis met with mixed results. As expected, 677
621 (S1: µ = −0.688, SE = 0.158; S2: µ = −0.156, SE = 0.169). An in- G-SYNC’s effects were fewer than in Watson et al., but G-SYNC 678
622 teraction of SN with GE (Figure 5) showed that SN ’s effect was did not affect the SAM emotion measures at all. We were correct 679
623 focused in novice players: while they became more excited during in predicting that G-SYNC would have no or minor effects on the 680
624 the first session of gameplay, they became slightly calmer during SDA measure of subjective difficulty. Finally, our expectation that 681
625 the second (S1: µ = −0.813, SE = 0.228; S2: µ = 0.250, SE = 0.194; p G-SYNC would affect our IAT engagement measure slightly proved 682
626 = 0.01, d = -1.257). On the other hand, veteran players experienced correct, at least for those using G-SYNC in the second session. 683
627 684

6
 Am I Playing Better Now? A Study of Effects of Adaptive Synchronization in 60Hz Gameplay Woodstock ’18, June 03–05, 2018, Woodstock, NY

685 742
686 743
687 744
688 745
689 746
690 747
691 748
692 749
693 750
694 751
695 752
696 753
697 754
698 755
699 756
700 757
701 758
702 Fig. 6. Procedure for our second G-SYNC between participants experiment, and how it relates to independent variables and dependent measures. Boldface 759
703 indicates independent variables, while blue blocks represent gameplay with G-SYNC on, red blocks with G-SYNC off. 760
704 761
705 762
706 4.7.2 Session and Expertise Effects. Several effects of session and playing both sessions either with or without G-SYNC. Figure 6 763
707 expertise offer reassurance that our measures are sensitive to partici- shows how this affected our overall procedure, with G-SYNC on in 764
708 pant performance and experience, despite G-SYNC’s limited impact. light blue completely on the top, and G-SYNC off in pink completely 765
709 Veteran players outscored novices in the second session, when they on the bottom. Because each participant saw only one G-SYNC 766
710 knew the game better and had more scoring opportunities. This setting, we could not use the IAT to measure G-SYNC’s effect on 767
711 may account for the fact that only veterans continued to become engagement, and so did not use it as a dependent measure. Also, in 768
712 more excited through the second session. Similar effects were found our perceptibility measure, we could not ask them which session 769
713 by Watson et al. [61], though session and expertise impacted SAM used G-SYNC. Instead, we asked them if they believed they had used 770
714 valence and dominance rather than arousal. a G-SYNC display. 771
715 772
716
5 EXPERIMENT 2: G-SYNC BETWEEN PARTICIPANTS 5.2 Procedure and Task 773
717 774
Several significant results from our first experiment indicated that When participants arrived, we assigned half of them randomly into
718 775
B4 is less challenging and has less experiential impact at the game’s the G-SYNC on group and the other half into the G-SYNC off group.
719 776
beginning. This matches a common design pattern that eases players Then, they went through the same seven experimental stages (Figure
720 777
into games, allowing them to enjoy and succeed in early gameplay, 5), the exception being that GSYNC was never toggled during their
721 778
while also gaining the skill and familiarity with both game and gameplay. Participants took 45 minutes to complete the experiment
722 779
platform that permits similar success later in the game. on average (SE = 1.1).
723 780
724
Our first experiment’s within-participant design increased sta- 781
tistical power and permitted each participant to compare displays 5.3 Hypotheses
725 782
with and without G-SYNC. However, its mid-gameplay switch of We based our hypotheses for G-SYNC’s effects in this second exper-
726 783
display technology certainly interfered with learning of the gam- iment primarily on the results of our first.
727 784
ing platform, and may also have interfered with learning of the
728 5.3.1 Perceptibility. Participants could not reliably perceive G-SYNC 785
game itself, just as that learning was most important. Moreover,
729 in the first experiment. In addition, in this experiment participants 786
such display technology changes are not typical of actual gameplay.
730 could not compare the two displays directly. We therefore hypothe- 787
For these reasons, we decided to replicate our experiment using a
731 sized that G-SYNC would again: 788
732
between-participant design, with each participant using just one 789
display technology, either with or without G-SYNC. Be imperceptible. Participants would not be able to identify
733 790
In the following three subsections, because of the many simi- whether they had used G-SYNC reliably.
734 791
larities between this experiment and our first, we note only the
735 5.3.2 Performance. Watson et al.’s results [61] show that G-SYNC 792
differences between the two studies.
736 can increase points earned in B4 at 30Hz frame rates, while in our 793
737 first experiment, G-SYNC had no effects at 60Hz. In this experiment, 794
738 5.1 Design and Variables participants will have more exposure to and practice with G-SYNC 795
739 We again used a 3-factor mixed design (2GS×2SN×2GE). In this case than they did in the first experiment. For this reason we again 796
740 however, G-SYNC was between-participants, with each participant hypothesized that G-SYNC would: 797
741 798

7
 Woodstock ’18, June 03–05, 2018, Woodstock, NY Anon.

799 856
800 857
801 858
802 859
803 860
804 861
805 862
806 863
807 864
808 865
809 866
810 867
811 868
812 869
813 870
814 871
Fig. 7. Experiment 2: Novice performance in the G-SYNC Between Partici- Fig. 8. Experiment 2: Veteran performance in the G-SYNC Between Partici-
815 872
pants study, and how it was affected by G-SYNC and Session. Effects were pants study, and how it was affected by G-SYNC and Session. Effects were
816 873
insignificant and negligible. Bars show standard error. larger and significantly different from the effects on novices. Bars show
817 874
standard error.
818 875
819 876
820 877
Improve performance slightly. Participants would score a few
821 878
more points with G-SYNC than without it, particularly in the
822
second gaming session, when they had gained more familiar- SN and GE had significantly related effects on scores. While 879
823
ity with G-SYNC. novices were unable to improve their scores in the second session 880
824 (S1: µ = 1763.8, SE = 151.2; S2: µ = 1922.5, SE = 287.5; p = .9411, d = 881
825 5.3.3 Experience. In our first experiment, G-SYNC affected only 0.155), veterans were (S1: µ = 2036.3, SE = 57.5; S2: µ = 4478.8, SE 882
826 the IAT measure of engagement, which we could not use in this = 352.3; p < .0001, d = 2.164). Finally, the interaction between SN 883
827 study. We therefore expected that G-SYNC would: and GE itself depended significantly on G-SYNC, creating a three- 884
828
Have no impact on experience, in SDA or any SAM measure. way interaction (F(1,36) = 6.43, p = 0.015, η 2 = 0.06). Novice players 885
829 (Figure 7) did not improve their scores significantly in the second 886
830 5.4 Results gameplay session, whether or not they used G-SYNC. On the other 887
831
We again report effects on perceptibility, performance, and expe- hand, veteran players (Figure 8) did improve their second session 888
832
rience, discussing significant (p < .05) and marginal (p < .1) main scores, with G-SYNC allowing an additional marginally significant 889
833
effects and interactions, with interaction means compared using improvement (on: µ = 5192.5, SE = 180.7; off: µ = 3765.0, SE = 890
834
contrasts. We performed 3-factor between-participants ANOVAs 614.9; p = 0.0573, d = 0.9961). While this improvement was not large 891
835
with G-SYNC (GS), session (SN ) and gaming expertise (GE) as factors. statistically, it was quite meaningful in gameplay. 892
836 893
837 5.4.1 Perceptibility. Participants correctly identified G-SYNC dis- 5.5.2 Experience. G-SYNC had no effects on any of the SAM emo- 894
838 play 52% of the time, which was not significantly different than tional measures. However, session did affect SAM arousal signifi- 895
839 chance (50%) (χ 2 (1,40) = 0.025, p = .87). cantly: players gained more excitement from playing the first session 896
840 than from the second (S1: µ = −0.850, SE = 0.146; S2: µ = −0.250, 897
841 5.5 Participants SE = 0.138). Session and expertise combined to affect SAM domi- 898
842
To compensate for the reduced statistical power of a between- nance significantly (Figure 9). While novice players experienced 899
843
participants design, we recruited a larger number of computer sci- little change in control in the first session (µ = 0.100, SEM = 0.250) 900
844
ence students: 40, comprised of 20 novices (GE<5) and 20 veterans and a loss of control during the second session (µ = −0.350, SEM 901
845
(GE>5). = 0.196), veteran players experienced a loss of control in the first 902
846 session (µ = −0.300, SEM = 0.242), followed by a gain of control in 903
847 5.5.1 Performance. The effect of G-SYNC on B4 scores did not the second session (µ = 0.450, SEM = 0.170). 904
848 reach significance (on: µ = 2749.4, SE = 255.6; off: µ = 2351.3, Session also affected subjective difficulty and flow, as measured by 905
849 SE = 231.0; p = 0.137, η 2 = 0.04). Table 3 shows significant effects SDA. In the first gameplay session, participants overestimated actual 906
850 on performance. Scores did vary significantly across session, with time by a few (µ = 3.475, SE = 0.705) minutes, indicating that they 907
851 players averaging 1900 points in the first session (SE = 82.7), and experienced difficulty. In the second session, they overestimated 908
852 3200.6 in the second (SE = 303.7). Scores also varied significantly time by about one minute (µ = 1.150, SE = 0.952), experiencing much 909
853 with expertise: novice players (<5) averaged 1843.1 points (SE = less difficulty. A marginal three-way interaction also affected SDA 910
854 160.8), while veteran players averaged 3257.5 points (SE = 263.2). and subjective difficulty (see Figures 10 and 11). Novices experienced 911
855 912

8
 Am I Playing Better Now? A Study of Effects of Adaptive Synchronization in 60Hz Gameplay Woodstock ’18, June 03–05, 2018, Woodstock, NY

913 Table 3. ANOVAs of the second between-participants experiment 970

914 971
915 measure effects distribution p value η2 972
916 973
GP SN F(1,36) = 43.2 <0.001 = 0.3
917 974
GP GE F(1,36) = 29.2 <0.001 = 0.34
918 975
GP SN × GE F(1,36) = 33.3 <0.001 = 0.25
919 976
GP GS × SN × GE F(1,36) = 6.43 = 0.015 = 0.06
920 977
SAM-A SN F(1,36) = 5.77 = 0.021 = 0.11
921 978
SAM-D SN × GE F(1,36) = 6.24 = 0.017 = 0.09
922 979
SDA SN F(1,36) = 10.8 = 0.002 = 0.05
923 980
SDA GS × SN × GE F(1,36) = 3.5 = 0.069 = 0.02
924 981
925 982
926 983
927 984
G-SYNC effects we found were limited to components of these inter-
928 985
Fig. 9. Mean SAM-Dominance from our second G-SYNC between partici- actions: veteran players in the second session were able to increase
929 986
pants experiment, grouped by session and gaming expertise. Error bars show scoring when using G-SYNC, and this likely translated to their ex-
930 987
standard error periencing more difficulty in the second session with G-SYNC than
931 988
without it.
932 989
933 990
6 GENERAL DISCUSSION
934 991
the same level of difficulty in both sessions without G-SYNC. With G- Before we begin discussing our overall results in both experiments,
935 992
SYNC, they felt much less difficulty in the second session. Veterans’ we contextualize them by noting the limitations of our work. We
936 993
experience was nearly a mirror image: they experienced similar then examine several concrete, applied questions; and continue with
937 994
levels of difficulty in both sessions with G-SYNC, and without it, a discussion of a few more theoretical issues raised by our results.
938 995
much less difficulty in the second session.
939 996
940 6.1 Limitations 997
5.6 Discussion
941 Several aspects of our research suggest caution in interpreting and 998
942
We again begin considering these results with an examination of applying its results. With regard to participants, most were similarly 999
943
our hypotheses about G-SYNC’s effects, and continue with an ex- aged college students. Today’s gamers are much more diverse. Also, 1000
944
amination of session and expertise’s impacts. many of the effects we found in our second experiment were focused 1001
945
5.6.1 Hypotheses About G-SYNC’s Effects. We again verified our on only 10 veteran gamers using (or not using) G-SYNC in the second 1002
946
perceptibility hypothesis. Participants could not reliably determine session, limiting our statistical power. A a confirming experiment 1003
947
whether or not their display used G-SYNC, and therefore brought would be useful, with more veteran gamers to increase power. 1004
948
no cognitive biases about ASync to our experiential measures. We Considering design variables, Watson et al. [61] examined 30Hz 1005
949
also confirmed our slight improvement to performance hypothesis. gameplay, and we examined 60Hz play. However, there is a strong in- 1006
950
G-SYNC had no broad effect on B4 scores, but it did allow veterans dustry trend toward still higher frame rates especially among esports 1007
951
playing in the second, more challenging session to raise their scores athletes. In addition, as we briefly discussed earlier, we used B4’s 1008
952
by 38%. We found partial confirmation for our experience hypothesis. native scoring system to measure gaming performance. This was 1009
953
As we expected, G-SYNC did not affect the SAM emotion measures. subject to in-game variation in difficulty and scoring opportunities, 1010
954
However, it did have a marginally significant effect on subjective making it difficult to draw conclusions about participant learning, 1011
955
difficulty as measured by SDA. and to generalize these results to other games. Study of high per- 1012
956 formance gaming would benefit from performance measures that 1013
957 5.6.2 Session and Expertise Effects. Session’s effects on performance allow comparison of game difficulty to learning, and different games 1014
958 and experience were extensive, and paralleled many of session’s ef- to one another. We experimented with scores normalized by max- 1015
959 fects in our first experiment. Players scored more points in the ima, and by scoring percentiles. Other possibilities might include 1016
960 second session of gameplay than in the first, likely due to the in- Z-scores and scoring rates (as opposed to scores themselves). 1017
961 creased scoring opportunities it offered, and the familiarity with With respect to apparatus and procedure, we used Battlefield 4 in 1018
962 the game gained during the first session. Players also gained more our work; a well-known example of the FPS gaming genre. Although 1019
963 excitement and experienced more difficulty in the first session than this genre is highly sensitive to latency, other gaming genres such as 1020
964 in the second. sports and role-playing merit study. Perhaps most importantly, our 1021
965 Expertise’s only main effect was on performance, with veteran study examined gameplay over a fairly short time (< 30 minutes). 1022
966 players outscoring novice players. However, expertise interacted Typical real world play sessions can span hours, and depending 1023
967 with session, with only veteran players able to increase their scoring on the gaming genre, players may continue their play over weeks, 1024
968 and sense of control in the second session of gameplay. The two months or even years. 1025
969 1026

9
 Woodstock ’18, June 03–05, 2018, Woodstock, NY Anon.

1027 1084
1028 1085
1029 1086
1030 1087
1031 1088
1032 1089
1033 1090
1034 1091
1035 1092
1036 1093
1037 1094
1038 1095
1039 1096
1040 1097
1041 1098
1042 1099
Fig. 10. Experiment 2: Novice subjective difficulty (SDA) in the G-SYNC Fig. 11. Experiment 2: Veteran subjective difficulty (SDA) in the G-SYNC
1043 1100
Between Participants study, and how it was affected by G-SYNC and Session. Between Participants study, and how it was affected by G-SYNC and Session.
1044 1101
Bars show standard error. Bars show standard error.
1045 1102
1046 1103
1047 1104
1048 6.2 Applied Questions 6.3.1 ...Playing Time. ASync’s game performance benefits may in- 1105
1049 We continue our discusson by examining applied questions that are crease with playing time. In our first experiment, participants used 1106
1050 easy to pose, but may have surprisingly complex answers. G-SYNC for only 10 to 15 minutes, and G-SYNC did not improve gam- 1107
1051 ing performance. In our second study, participants using G-SYNC 1108
1052 6.2.1 Can players tell when G-SYNC is in use? In general, no. In 1109
did so for 20-25 minutes, and we found that G-SYNC improved
1053 Watson et al.’s [61] examination of ASync with 30Hz gameplay, par- 1110
veteran’s scores in the second session of gameplay. Increased play-
1054 ticipants playing both with and without G-SYNC could not reliably 1111
ing time allows gamers not only to learn about the game and its
1055 differentiate between them. Our two experiments studied ASync 1112
challenges, but also about ASync itself, and its impact on gameplay,
1056 with 60Hz gameplay, when it has still less measurable temporal 1113
allowing gamers to exploit ASync’s potential benefits more fully.
1057 impact, and again found that participants could not reliably identify 1114
1058 G-SYNC. However, it is still possible that extended play or high 6.3.2 ...Expertise. ASync’s performance and experiential impacts 1115
1059 expertise may enable gamers to identify ASync reliably. vary with gamer expertise. In our second experiment’s second ses- 1116
1060 sion, only veteran gamers were able to exploit G-SYNC to increase 1117
6.2.2 Does ASync improve gaming performance? It can, as demon-
1061 their score. ASync’s performance advantages may be largest for 1118
strated by Watson et al. [61], and to a more limited extent by our
1062 expert gamers. In both of our experiments, measures of experience 1119
work here. Because ASync’s impact on performance in our 60Hz ex-
1063 regularly varied with gamer expertise. We suspect that novice and 1120
periments was more limited than in Watson et al.’s 30Hz experiment,
1064 veteran gamers evaluate gameplay differently. For example, veteran 1121
we expect that ASync’s impact on gaming performance lessens as
1065 gamers may value gaming performance more than novice gamers, 1122
frame rates rise and latencies fall. We also found evidence suggest-
1066 resulting in different gaming experiences. 1123
ing that ASync’s impact on gaming performance depends on gamer
1067 1124
expertise, playing time, and game content. For more detail on these 6.3.3 ...Game Content. ASync’s performance benefits likely depend
1068 1125
relationships, see below. on the nature of a game’s challenges. Our experiments did not ex-
1069 1126
plicitly control those challenges, but our second experiment showed
1070 6.2.3 Does ASync improve gaming experience? It can, though gam- 1127
that G-SYNC’s performance benefits were limited to the second
1071 ing experience itself is complex, and highly dependent on individ- 1128
session of gameplay. Some of these benefits relate to the gamer
1072 uals and context. In these experiments, G-SYNC increased gamer 1129
learning permitted by increased playing time upon reaching the
1073 engagement slightly, and sometimes improved gamers’ self-reported 1130
second session, but some benefits may also result from the nature
1074 feelings of control and excitement. These improvements usually de- 1131
of the game’s challenges during the second session. In particular,
1075 pended on whether gamers were novices or veterans, and what 1132
tasks that require rapid response or continuous monitoring may be
1076 part the game they were playing. For example in our second experi- 1133
particularly sensitive to the latencies that ASync reduces.
1077 ment, novices lost felt control in the second part of the game, while 1134
1078 veterans gained in felt control. 6.3.4 ...How it Helps. Finally, we consider how ASync impacts 1135
1079 performance and experience. Our second experiment showed that 1136
1080 6.3 Issues and Implications while both novice and veteran experience were affected by G-SYNC, 1137
1081 Our work raises several interesting possibilities that merit discus- only veteran gamers were able to realize any of G-SYNC’s perfor- 1138
1082 sion, and perhaps further research. If we consider ASync and... mance benefits. It may be that novice gamers benefit primarily 1139
1083 1140

10
 Am I Playing Better Now? A Study of Effects of Adaptive Synchronization in 60Hz Gameplay Woodstock ’18, June 03–05, 2018, Woodstock, NY

1141 from ASync’s advantages for perception, which change their expe- [10] Sven Charleer, Kathrin Gerling, Francisco Gutiérrez, Hans Cauwenbergh, Bram 1198
1142 rience; while veteran gamers also benefit from ASync’s advantages Luycx, and Katrien Verbert. 2018. Real-time dashboards to support esports spectat- 1199
ing. In Proceedings of the 2018 Annual Symposium on Computer-Human Interaction
1143 for action, which permit them to realize improvements in game in Play. 59–71. 1200
1144 performance. [11] Jessie YC Chen and Jennifer E Thropp. 2007. Review of low frame rate effects on 1201
1145
human performance. IEEE Transactions on Systems, Man, and Cybernetics-Part A: 1202
Systems and Humans 37, 6 (2007), 1063–1076.
1146 7 FUTURE WORK [12] Mark Claypool and Kajal Claypool. 2006. On latency and player actions in online 1203
1147 games. (2006). 1204
Research on ASync’s impacts is sparse, and there are many promis- [13] Mark Claypool and Kajal Claypool. 2010. Latency can kill: precision and deadline
1148 1205
ing directions for further research. We encourage research address- in online games. In Proceedings of the first annual ACM SIGMM conference on
1149 Multimedia systems. ACM, 215–222. 1206
ing the limitations of our work, including increasing diversity among [14] Mark Claypool, Andy Cockburn, and Carl Gutwin. 2019. Game input with delay:
1150 1207
participants and gaming genres. In addition, it would be interesting moving target selection parameters. In Proceedings of the 10th ACM Multimedia
1151 Systems Conference. ACM, 25–35. 1208
to compare the performance and experiential impacts of FreeSync
1152 [15] Mark Claypool, Ragnhild Eg, and Kjetil Raaen. 2016. The effects of delay on 1209
vs G-SYNC, as well as ASync, VSync and no VSync (with tearing). game actions: Moving target selection with a mouse. In Proceedings of the 2016
1153 1210
We also advocate research investigating the issues we raised in Annual Symposium on Computer-Human Interaction in Play Companion Extended
1154 Abstracts. ACM, 117–123. 1211
our general discussion, including ASync’s benefits as playing time
1155 [16] Tom Cole, Paul Cairns, and Marco Gillies. 2015. Emotional and functional chal- 1212
and gamer expertise increases. In particular, study of the extreme lenge in core and avant-garde games. In Proceedings of the 2015 annual symposium
1156 1213
expertise of e-sports athletes and their temporal needs should be on computer-human interaction in play. 121–126.
1157 [17] Mary Czerwinski, Eric Horvitz, and Edward Cutrell. 2001. Subjective duration 1214
informative. More broadly, research on high-performance interfaces assessment: An implicit probe for software usability. In Proceedings of IHM-HCI
1158 1215
of all kinds is sparse. As computing technology reaches more deeply 2001 conference, Vol. 2. 167–170.
1159 [18] Jonathan Deber, Ricardo Jota, Clifton Forlines, and Daniel Wigdor. 2015. How 1216
into our work and play, the need for interfaces supporting excep-
1160 much faster is fast enough?: User perception of latency & latency improvements 1217
tional performance will continue to grow. in direct and indirect touch. In Proceedings of the 33rd Annual ACM Conference on
1161 1218
Human Factors in Computing Systems. ACM, 1827–1836.
1162 [19] Thierry Devos and Mahzarin R Banaji. 2005. American= white? Journal of 1219
1163
8 CONCLUSION personality and social psychology 88, 3 (2005), 447. 1220
ASync lowers latencies and increases frame rates by improving [20] Matthias Dick, Oliver Wellnitz, and Lars Wolf. 2005. Analysis of factors affecting
1164 1221
players’ performance and perception in multiplayer games. In Proceedings of 4th
1165 synchronization between displays and GPUs. This study builds on ACM SIGCOMM workshop on Network and system support for games. ACM, 1–7. 1222
1166 prior work examining ASync’s effects on gaming performance and [21] Boris Egloff and Stefan C Schmukle. 2002. Predictive validity of an implicit 1223
association test for assessing anxiety. Journal of personality and social psychology
1167 experience by examining ASync’s effects at a higher 60Hz average 83, 6 (2002), 1441. 1224
1168 frame rate preferred by gamers. We find that while ASync’s overall [22] Chris Ellis, Syed Zain Masood, Marshall F Tappen, Joseph J LaViola, and Rahul 1225
1169 impact is less at 60Hz than at 30Hz, experiential impacts remain, and Sukthankar. 2013. Exploring the trade-off between accuracy and observational 1226
latency in action recognition. International Journal of Computer Vision 101, 3
1170 gaming performance benefits are still available to veteran gamers. (2013), 420–436. 1227
1171 [23] Katharina Emmerich, Patrizia Ring, and Maic Masuch. 2018. I’m Glad You Are 1228
on My Side: How to Design Compelling Game Companions. In Proceedings of the
1172 9 ACKNOWLEDGMENTS 2018 Annual Symposium on Computer-Human Interaction in Play. 141–152.
1229
1173 [24] Fraps. 2013. FRAPS game capture video recorder fps viewer. http://www.fraps. 1230
Removed for anonymous review.
1174 com/. (Accessed on 08/29/2019). 1231
[25] Julian Frommel, Fabian Fischbach, Katja Rogers, and Michael Weber. 2018.
1175 Emotion-based Dynamic Difficulty Adjustment Using Parameterized Difficulty 1232
REFERENCES
1176 and Self-Reports of Emotion. In Proceedings of the 2018 Annual Symposium on 1233
[1] Sultan A Alharthi, Olaa Alsaedi, Zachary O Toups, Joshua Tanenbaum, and Jessica Computer-Human Interaction in Play. 163–171.
1177 Hammer. 2018. Playing to wait: A taxonomy of idle games. In Proceedings of the 1234
[26] Julian Frommel, Claudia Schrader, and Michael Weber. 2018. Towards Emotion-
1178 2018 CHI Conference on Human Factors in Computing Systems. 1–15. based Adaptive Games: Emotion Recognition Via Input and Performance Features. 1235
[2] AMD. 2019. Radeon FreeSyncTechnology | FreeSync 2 HDR Games | AMD. https: In Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in
1179 1236
//www.amd.com/en/technologies/free-sync. (Accessed on 08/19/2019). Play. 173–185.
1180 [3] Michelle Annett, Albert Ng, Paul Dietz, Walter F. Bischof, and Anoop Gupta. 2014. 1237
[27] Video game developer EA DICE and Electronic Arts. 2013. Battlefield. Game[B4].
1181 How Low Should We Go?: Understanding the Perception of Latency While Inking. [28] Chad Phoenix Rose Gowler and Ioanna Iacovides. 2019. " Horror, guilt and 1238
In Proceedings of Graphics Interface 2014 (GI ’14). Canadian Information Processing shame"–Uncomfortable Experiences in Digital Games. In Proceedings of the Annual
1182 Society, Toronto, Ont., Canada, Canada, 167–174. http://dl.acm.org/citation.cfm? 1239
Symposium on Computer-Human Interaction in Play. 325–337.
1183 id=2619648.2619677 [29] Anthony G Greenwald and Shelly D Farnham. 2000. Using the implicit association 1240
1184
[4] Benjamin B Bederson. 2004. Interfaces for staying in the flow. Ubiquity 2004, test to measure self-esteem and self-concept. Journal of personality and social 1241
September (2004), 1–1. psychology 79, 6 (2000), 1022.
1185 [5] Tom Beigbeder, Rory Coughlan, Corey Lusher, John Plunkett, Emmanuel Agu, 1242
[30] Anthony G Greenwald, Debbie E McGhee, and Jordan LK Schwartz. 1998. Mea-
1186 and Mark Claypool. 2004. The effects of loss and latency on user performance suring individual differences in implicit cognition: the implicit association test. 1243
in unreal tournament 2003®. In Proceedings of 3rd ACM SIGCOMM workshop on Journal of personality and social psychology 74, 6 (1998), 1464.
1187 1244
Network and system support for games. ACM, 144–151. [31] Anthony G Greenwald, Brian A Nosek, and Mahzarin R Banaji. 2003. Under-
1188 [6] Julia Ayumi Bopp, Elisa D Mekler, and Klaus Opwis. 2016. Negative emotion, 1245
standing and using the implicit association test: I. An improved scoring algorithm.
1189 positive experience? Emotionally moving moments in digital games. In Proceedings Journal of personality and social psychology 85, 2 (2003), 197. 1246
of the 2016 CHI Conference on Human Factors in Computing Systems. 2996–3006. [32] Anthony G Greenwald, T Andrew Poehlman, Eric Luis Uhlmann, and Mahzarin R
1190 [7] Julia Ayumi Bopp, Livia J Müller, Lena Fanya Aeschbach, Klaus Opwis, and Elisa D 1247
Banaji. 2009. Understanding and using the Implicit Association Test: III. Meta-
1191 Mekler. 2019. Exploring Emotional Attachment to Game Characters. In Proceedings analysis of predictive validity. Journal of personality and social psychology 97, 1 1248
1192
of the Annual Symposium on Computer-Human Interaction in Play. 313–324. (2009), 17. 1249
[8] Julia Ayumi Bopp, Klaus Opwis, and Elisa D Mekler. 2018. “An Odd Kind of [33] Juho Hamari and Max Sjöblom. 2017. What is eSports and why do people watch
1193 Pleasure” Differentiating Emotional Challenge in Digital Games. In Proceedings of 1250
it? Internet research (2017).
1194 the 2018 chi conference on human factors in computing systems. 1–12. [34] Oliver Hohlfeld, Hannes Fiedler, Enric Pujol, and Dennis Guse. 2016. Insensitivity 1251
[9] Margaret M Bradley and Peter J Lang. 1994. Measuring emotion: the self- to Network Delay: Minecraft Gaming Experience of Casual Gamers. In 2016 28th
1195 1252
assessment manikin and the semantic differential. Journal of behavior therapy International Teletraffic Congress (ITC 28), Vol. 3. IEEE, 31–33.
1196 and experimental psychiatry 25, 1 (1994), 49–59. 1253
1197 1254

11
 Woodstock ’18, June 03–05, 2018, Woodstock, NY Anon.

1255 [35] Katherine Isbister. 2006. Better game characters by design: A psychological approach. [62] AF Wattimena, Robert E Kooij, JM Van Vugt, and OK Ahmed. 2006. Predicting the 1312
1256 CRC Press. perceived quality of a first person shooter: the Quake IV G-model. In Proceedings 1313
[36] Katherine Isbister. 2016. How games move us: Emotion by design. Mit Press. of 5th ACM SIGCOMM workshop on Network and system support for games. ACM,
1257 [37] Zenja Ivkovic, Ian Stavness, Carl Gutwin, and Steven Sutcliffe. 2015. Quantifying 42. 1314
1258 and mitigating the negative effects of local latencies on aiming in 3d shooter [63] Eric N Wiebe, Allison Lamb, Megan Hardy, and David Sharek. 2014. Measur- 1315
1259
games. In Proceedings of the 33rd Annual ACM Conference on Human Factors in ing engagement in video game-based environments: Investigation of the User 1316
Computing Systems. ACM, 135–144. Engagement Scale. Computers in Human Behavior 32 (2014), 123–132.
1260 [38] Benjamin F Janzen and Robert J Teather. 2014. Is 60 FPS better than 30?: the 1317
1261 impact of frame rate and latency on moving target selection. In Proceedings of 1318
the extended abstracts of the 32nd annual ACM conference on Human factors in
1262 1319
computing systems. ACM, 1477–1482.
1263 [39] Ricardo Jota, Albert Ng, Paul Dietz, and Daniel Wigdor. 2013. How fast is fast 1320
1264 enough?: a study of the effects of latency in direct-touch pointing tasks. In Pro- 1321
ceedings of the sigchi conference on human factors in computing systems. ACM,
1265 2291–2300. 1322
1266 [40] J Kampman. 2017. Poll: What’s the resolution and refresh rate of your gaming 1323
1267
monitor? 1324
[41] Carney Landis. 1954. Determinants of the critical flicker-fusion threshold. Physi-
1268 ological Reviews 34, 2 (1954), 259–286. 1325
1269 [42] Kristin A Lane, Mahzarin R Banaji, Brian A Nosek, and Anthony G Greenwald. 1326
2007. Understanding and using the implicit association test: IV. Implicit measures
1270 1327
of attitudes (2007), 59–102.
1271 [43] John Alan Lee. 1977. A typology of styles of loving. Personality and Social 1328
1272 Psychology Bulletin 3, 2 (1977), 173–182. 1329
[44] Michael Sangyeob Lee and Carrie Heeter. 2015. Cognitive Intervention and
1273 Reconciliation: NPC Believability in Single-Player RPGs. International Journal of 1330
1274 Role-Playing 5 (2015), 47–65. 1331
1275
[45] Steven WK Lee and Rocky KC Chang. 2018. Enhancing the experience of multi- 1332
player shooter games via advanced lag compensation. In Proceedings of the 9th
1276 ACM Multimedia Systems Conference. ACM, 284–293. 1333
1277 [46] Changchun Liu, Pramila Agrawal, Nilanjan Sarkar, and Shuo Chen. 2009. Dynamic 1334
difficulty adjustment in computer games through real-time anxiety-based affective
1278 1335
feedback. International Journal of Human-Computer Interaction 25, 6 (2009), 506–
1279 529. 1336
1280 [47] Michael Long and Carl Gutwin. 2018. Characterizing and Modeling the Effects of 1337
Local Latency on Game Performance and Experience. In Proceedings of the 2018
1281 Annual Symposium on Computer-Human Interaction in Play. ACM, 285–297. 1338
1282 [48] Michael Long and Carl Gutwin. 2019. Effects of Local Latency on Game Pointing 1339
1283
Devices and Game Pointing Tasks. In Proceedings of the 2019 CHI Conference on 1340
Human Factors in Computing Systems. ACM, 208.
1284 [49] Adam W Meade. 2009. FreeIAT: An open-source program to administer the 1341
1285 Implicit Association Test. Applied psychological measurement (2009). 1342
[50] Albert Mehrabian and James A Russell. 1974. An approach to environmental
1286 1343
psychology. the MIT Press.
1287 [51] Jeanne Nakamura and Mihaly Csikszentmihalyi. 2014. The concept of flow. In 1344
1288 Flow and the foundations of positive psychology. Springer, 239–263. 1345
[52] Newzoo. 2019. Global Esports Economy Will Top $ 1 Billion for the First Time in
1289 2019. (2019). 1346
1290 [53] Newzoo. 2019. New Twitch Rankings: Top Games by Esports and Total Viewing 1347
1291
Hours. (2019). 1348
[54] NVIDIA. 2019. G-SYNC Ultimate Gaming Monitors. https://www.nvidia.com/en-
1292 us/geforce/products/g-sync-monitors/. (Accessed on 08/19/2019). 1349
1293 [55] Christian H Poth, Rebecca M Foerster, Christian Behler, Ulrich Schwanecke, 1350
Werner X Schneider, and Mario Botsch. 2018. Ultrahigh temporal resolution of
1294 1351
visual presentation using gaming monitors and G-Sync. Behavior research methods
1295 50, 1 (2018), 26–38. 1352
1296 [56] Shaghayegh Roohi, Elisa D Mekler, Mikke Tavast, Tatu Blomqvist, and Perttu 1353
Hämäläinen. 2019. Recognizing Emotional Expression in Game Streams. In Pro-
1297 ceedings of the Annual Symposium on Computer-Human Interaction in Play. 301– 1354
1298 311. 1355
1299
[57] Samit Sarkar. 2014. Why frame rate and resolution matter: A graphics primer. Poly- 1356
gon (2014). https://www.polygon.com/2014/6/5/5761780/frame-rate-resolution-
1300 graphics-primer-ps4-xbox-one 1357
1301 [58] Josef Spjut, Ben Boudaoud, Kamran Binaee, Jonghyun Kim, Alexander Majercik, 1358
Morgan McGuire, David Luebke, and Joohwan Kim. 2019. Latency of 30 ms
1302 1359
Benefits First Person Targeting Tasks More Than Refresh Rate Above 60 Hz. In
1303 SIGGRAPH Asia 2019 Technical Briefs. 110–113. 1360
1304 [59] Samuel Stewart. 2019. What Is The Best FPS For Gaming? Gaming Scan (2019). 1361
https://www.gamingscan.com/best-fps-gaming/
1305 [60] Andrew B Watson and Albert J Ahumada. 2011. 64.3: flicker visibility: a perceptual 1362
1306 metric for display flicker. In SID symposium digest of Technical Papers, Vol. 42. 1363
1307
Wiley Online Library, 957–959. 1364
[61] Benjamin Watson, Rachit Shrivastava, and Ajinkya Gavane. 2019. The Effects
1308 of Adaptive Synchronization on Performance and Experience in Gameplay. Pro- 1365
1309 ceedings of the ACM on Computer Graphics and Interactive Techniques 2, 1 (2019), 1366
5.
1310 1367
1311 1368

12