Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more
Download
Standard view
Full view
of .
Look up keyword
Like this
3Activity
0 of .
Results for:
No results containing your search query
P. 1
Using Cursor Movements to Understand and Improve Search

Using Cursor Movements to Understand and Improve Search

Ratings: (0)|Views: 3,593 |Likes:
Published by E-Siber
Understanding how people interact with search engines is
important in improving search quality. Web search engines
typically analyze queries and clicked results, but these actions
provide limited signals regarding search interaction.
Laboratory studies often use richer methods such as gaze
tracking, but this is impractical at Web scale. In this paper,
mouse cursor is examined behavior on search engine results
pages (SERPs), including not only clicks but also cursor
movements and hovers over different page regions.
Authors: Jeff Huang, Ryen W. White and Susan Dumais.
Understanding how people interact with search engines is
important in improving search quality. Web search engines
typically analyze queries and clicked results, but these actions
provide limited signals regarding search interaction.
Laboratory studies often use richer methods such as gaze
tracking, but this is impractical at Web scale. In this paper,
mouse cursor is examined behavior on search engine results
pages (SERPs), including not only clicks but also cursor
movements and hovers over different page regions.
Authors: Jeff Huang, Ryen W. White and Susan Dumais.

More info:

Published by: E-Siber on May 23, 2011
Copyright:Attribution Non-commercial

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less

05/26/2011

pdf

text

original

 
 
No Clicks, No Problem: Using Cursor Movementsto Understand and Improve Search
Jeff Huang
Information SchoolUniversity of Washingtonchi@jeffhuang.com
Ryen W. White
Microsoft ResearchRedmond, WA 98052ryenw@microsoft.com
Susan Dumais
Microsoft ResearchRedmond, WA 98052sdumais@microsoft.com
ABSTRACT
Understanding how people interact with search engines isimportant in improving search quality. Web search enginestypically analyze queries and clicked results, but these ac-tions provide limited signals regarding search interaction.Laboratory studies often use richer methods such as gazetracking, but this is impractical at Web scale. In this paper,we examine mouse cursor behavior on search engine resultspages (SERPs), including not only clicks but also cursormovements and hovers over different page regions. We: (i)report an eye-tracking study showing that cursor position isclosely related to eye gaze, especially on SERPs
;
(ii) pre-sent a scalable approach to capture cursor movements, andan analysis of search result examination behavior evident inthese large-scale cursor data
;
and (iii) describe two applica-tions (estimating search result relevance and distinguishinggood from bad abandonment) that demonstrate the value of capturing cursor data. Our findings help us better under-stand how searchers use cursors on SERPs and can helpdesign more effective search systems. Our scalable cursortracking method may also be useful in non-search settings.
Author Keywords
Cursor movements, clicks, implicit feedback, Web search.
ACM Classification Keywords
H.3.3 [Information Storage and Retrieval]: InformationSearch and Retrieval–
selection process, relevance feedback 
General Terms
Experimentation, Human Factors, Measurement.
INTRODUCTION
Understanding how people interact with Web sites is im-portant in improving site design and the quality of servicesoffered. The Web provides unprecedented opportunities toevaluate alternative design, interaction, and algorithmicmethods at scale and
in situ
with actual customers doingtheir own tasks in their own environments [19]. Such stud-ies typically involve measuring clicks which can be ob-tained easily at scale. However, they fail to capture behav-iors that do not lead to clicks (e.g., which items are attend-ed to, in what order, etc.) or subjective impressions. Gaze-tracking studies with participants present in the laboratorycan provide more detailed insights but on a smaller scale.In this paper we consider how mouse movements, whichcan be collected remotely on a large scale, can be used tounderstand richer patterns of behavior.We focus on understanding cursor activities in Web searchbehavior. People conduct Web searches to satisfy infor-mation needs. Their interaction with search engines beginsby issuing a search query, then reviewing the search engineresults page (SERP) to determine which, if any, results maysatisfy their need. In doing so, they may move their mousecursor around the page, hovering over and possibly clickingon hyperlinks. Small-scale laboratory studies have ob-served participants making many uses of the cursor onSERPs beyond hyperlink clicking [1,21,25]. These usesinclude moving the cursor as a reading aid, using it to mark interesting results, using it to interact with controls on thescreen (e.g., buttons, scroll bars), or simply positioning thecursor so that it does not occlude Web page content. How-ever, studying such behaviors in small-scale laboratorysettings is limited in terms of what inferences can be made.Tracking mouse cursor movements
at scale
can provide arich new source of behavioral information to understand,model, and satisfy information needs. Recent research hasshown that cursor movements correlate with eye gaze[6,13,25,26], and may therefore be an effective indicator of user attention. We believe that cursor data, like click data[18], can provide signals that reveal searcher intent andmay be useful in improving the search experience. Cursordata can be used to complement click data in several ways.First, cursor data can be captured for uncommon querieswhere strong indicators of relevance such as result clicksmay occur less frequently or not at all. For example, ana-lyzing click logs for a query that has been issued severaltimes but never clicked may provide limited relevance in-formation, but cursor behavior on the SERP associated withthe query may provide insight about relevance. Second, incases of so-called
good abandonment 
[20], where the con-tent on the SERP satisfies the user’s information need di-rectly, a search result click may be unnecessary. Thus thelack of a click should not always be interpreted as a searchfailure. Cursor behavior may help in distinguishing be-tween good and bad search abandonment.
Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise,or republish, to post on servers or to redistribute to lists, requires priorspecific permission and/or a fee.
CHI 2011,
May 7–12, 2011, Vancouver, BC, Canada.Copyright 2011 ACM 978-1-4503-0267-8/11/05....$10.00.
 
 
The research questions that we ask are: (i) to what extentdoes gaze correlate with cursor behavior on SERPs andnon-SERPs? (ii) what does cursor behavior reveal aboutsearch engine users’ result examination strategies, and howdoes this relate to search result clicks and prior eye-tracking research? and (iii) can we demonstrate useful ap-plications of large-scale cursor data? Answers to thesequestions help us determine the utility of cursor tracking atscale, and ultimately inform search system design and im-prove the experience for users of search engines.
RELATED WORK
One line of related research has explored the use of cursormovements, clicks, and gaze as
implicit indicators of inter-est 
on Web pages. In early work, Goecks and Shavlik modi-fied a Web browser to record themselves browsing hun-dreds of Web pages [11]. They found that a neural network could predict variables such as the amount of cursor activi-ty on the SERP, which they considered surrogate measure-ments of user interest. Claypool et al. [7] developed the“curious browser,” a custom Web browser that recordedactivity from 75 students browsing over 2,500 Web pages.They found that cursor travel time was a positive indicatorof a Web page’s relevance, but could only differentiatehighly irrelevant Web pages. Surprisingly, they also foundthat the number of mouse clicks on a page did not correlatewith its relevance. Hijikata [15] used client-side logging tomonitor five subjects browsing a total of 120 Web pages.They recorded actions such as text tracing and link pointingusing the cursor. The findings showed that these behaviorswere good indicators for interesting regions of the Webpage, around one-and-a-half times more effective than ru-dimentary term matching between the query and regions of the page. Shapira et al. [27] developed a special Webbrowser and recorded cursor activity from a small numberof company employees browsing the Web. They found thatthe ratio of mouse movement to reading time was a betterindicator of page quality than cursor travel distance andoverall length of time that users spend on a page.In the search domain, Guo and Agichtein [12] capturedmouse movements using a modified browser toolbar andfound differences in cursor travel distances between infor-mational and navigational queries. Furthermore, a decisiontree could classify the query type using cursor movementsmore accurately than using clicks. Guo and Agichtein alsoused interactions such as cursor movement, hovers, andscrolling to accurately infer search intent and interest insearch results [13]. They focused on automatically identify-ing a searcher’s research or purchase intent based on fea-tures of the interaction. Buscher et al. investigated the useof gaze tracking to predict salient regions of Web pages [2]and the use of visual attention as implicit relevance feed-back to personalize search [4].Another line of research examined the
relationship betweeneye gaze and cursor positions
. An early study by Chen etal. [6] measured this relationship in Web browsing by re-cording 100 gaze and cursor positions from five subjectsbrowsing the Web. They showed that the distance betweengaze and cursor was markedly shorter in regions of encoun-tered pages to which users attended. Liu and Chung [21]recorded cursor activity from 28 students browsing theWeb. They noticed patterns of cursor behaviors, includingreading by tracing text. Their algorithms were capable of predicting users’ cursor behaviors with 79% accuracy.More recent work has focused on the relationship betweencursor and gaze on search tasks. In a study involving 32subjects performing 16 search tasks each [25,26], Roddenet al. identified a strong alignment between cursor and gazepositions. They found that the distance between cursor andgaze positions was longer along the -axis than the -axis,and was generally shorter when the cursor was placed overthe search results. Rodden et al. also observed four generaltypes of mouse behaviors: neglecting the cursor while read-ing, using the cursor as a reading aid to follow text (eitherhorizontally or vertically), and using the cursor to mark interesting results. Guo and Agichtein [14] reported similarfindings in a smaller study of ten subjects performing 20search tasks each. Like Rodden et al., Guo and Agichteinnoticed that distances along the -axis tended to be longerthan the distances along the -axis. They could predict with77% accuracy when gaze and cursor were strongly alignedusing cursor features.The research presented in this paper extends previous work in a number of ways. Our analysis of the cursor-gaze rela-tionship (Study 1) involves more search tasks than priorstudies, compares SERP and post-SERP Web pages, andconfirms earlier results with a large study using the sameSERP layout that we use in the remainder of the paper.More importantly, we develop a scalable approach to cap-turing cursor data that enables us to analyze real user activi-ty in a natural setting for more than 360 thousand searchesfrom an estimated 22 thousand searchers (Study 2). Finally,using two case studies, we show how cursor data can sup-plement click data on two search-related problems.
STUDY 1: GAZE-CURSOR RELATIONSHIP
We begin by replicating and extending prior laboratoryexperiments on the relationship between gaze and cursoractivity using the same SERP layout deployed in our large-scale cursor study (Study 2, see Figure 2). Study 1 alsoinvolves more tasks and participants than prior laboratorystudies, and measures the relationship between gaze andcursor position on SERP and on non-SERP pages.
Data
We used a Tobii x50 eye tracker with 50Hz tracking fre-quency and 0.5° visual angle on a 1280
×
1024 resolution17 inch monitor (96.42dpi) and 1040
×
996 resolution In-ternet Explorer 7 browser. Cursor and gaze coordinateswere collected in an eye-tracking study of 38 participants(21 female, 17 male) performing Web searches. Participantswere recruited from a user study pool. They ranged in agebetween 26 and 60 years (mean = 45.5, = 8.2), and had awide variety of backgrounds and professions.
 
 
Each participant completed 32 search tasks on the samesearch engine, with the same SERP layout template, as usedfor the large
-
scale cursor study described in the next section(see Figure 2). Half of the tasks were navigational (i.e.,they had to find a specific Web page) and half were infor-mational (i.e., they had to find factual information). Eachtask started with a description of what participants shouldlook for on the Web. Gaze and cursor positions were rec-orded for each SERP as well as subsequent Web pages (i.e., pages visited after clicking on a search result). In total, wecollected data for 1,210 search tasks, 1,336,647 gaze posi-tions, and 87,227 cursor positions. Gaze
-
specific findingson this data set, unrelated to cursor behavior, have beenreported by others [
5
,10]. Those researchers granted usaccess to their data so that we could examine the relation-ship between gaze and cursor behaviors.
 
Gaze and cursor positions were extracted from the eye
-
tracking logs. In our data, the gaze positions were recordedapproximately every 20ms, whereas cursor positions wererecorded approximately every 100ms. Since cursor andgaze events did not always have identical timestamps, agaze position was interpolated for every cursor position.Interpolation was performed by calculating gaze andcoordinates weighted by the coordinates of the nearest gazecoordinates before and after the cursor position. For exam- ple, the interpolated
-
coordinate for eye gaze is calculatedas where is thetime for the corresponding cursor position, is the gaze’s
-
coordinate preceding the cursor position, recorded at time, and is the gaze’s
-
coordinate following the cursor  position, recorded at time . To reduce noise, cursor posi-tions were only captured if they occurred between gaze positions that were at most 100ms apart.
 
Findings
 
Figure 1 shows the frequency distribution for different val-ues of ∆ (distance between cursor 
-
coordinate and gaze
-
coordinate), (distance between cursor 
-
coordinateand gaze
-
coordinate), and Euclidean distance betweencursor and gaze coordinates, i.e.,
 
. Thesolid lines in Figure 1 show the distances for SERP pages.As can be seen, cursor and gaze positions are quite similar for both and
 y
values, their deltas peaking near 0, whenthe gaze and cursor positions are in the same place. Themean Euclidean distance between cursor and gaze is 178px(σ = 139px) and the median is 143px. The most commonoffset for the cursor is +3px (to the right) for the
-
coordinate and +29px (lower) for the
-
coordinate. That is,the cursor is most likely to be just below where the user isfocusing with their eyes. We also observed that the differ-ences are greater in the than direction (average 50px inthe direction and 7px in the direction), similar to other studies [14, 25]. Possible explanations for the difference between ∆ and ∆ include: (i) users may place the cursor to the left or right of their gaze to prevent it from obscuringthe text as they read up or down, and (ii) computer screensare usually wider, offering more horizontal space for thecursor.
 
The dotted lines in Figure 1 represent post
-
SERP landing pages. Distances between the gaze and cursor on the land-ing pages were greater than those on the SERP (215px vs.178px), perhaps due to greater variance in the layout andthe content of those pages, as has already been suggested by earlier gaze analysis [
2
]. Thus the cursor is a better  proxy for user attention on the SERP than post
-
SERP pag-es. Monitoring cursor behavior on SERPs may help esti-mate which results or features users attend to and when,and we now turn to a large
-
scale study of this.
 
Figure 1. ∆x, ∆y, and Euclidean distance plotted in a frequency distribution for SERP and post
-
SERP pages. Solid lines representthese distances gathered on the SERP, while dashed lines represented distances gathered on post
-
SERP pages (landing pages).
00.010.020.030.040.050.06
  -   6   0   0  -   5   7   0  -   5   4   0  -   5   1   0  -   4   8   0  -   4   5   0  -   4   2   0  -   3   9   0  -   3   6   0  -   3   3   0  -   3   0   0  -   2   7   0  -   2   4   0  -   2   1   0  -   1   8   0  -   1   5   0  -   1   2   0  -   9   0  -   6   0  -   3   00   3   0   6   0   9   0   1   2   0   1   5   0   1   8   0   2   1   0   2   4   0   2   7   0   3   0   0   3   3   0   3   6   0   3   9   0   4   2   0   4   5   0   4   8   0   5   1   0   5   4   0   5   7   0   6   0   0
   P  r  o  p  o  r   t   i  o  n  o   f  a   l   l  c  u  r  s  o  r  e  v  e  n   t  s
Distance (in pixels)SERP
¡ 
xSERP
¡ 
ySERP DistancePost-SERP
¡ 
xPost-SERP
¡ 
yPost-SERP Distance

You're Reading a Free Preview

Download
scribd
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->