Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more
Standard view
Full view
of .
Save to My Library
Look up keyword
Like this
0 of .
Results for:
No results containing your search query
P. 1
Understanding Social Tagging Systems

Understanding Social Tagging Systems

Ratings: (0)|Views: 19 |Likes:
Published by Umeverse

More info:

Published by: Umeverse on May 05, 2008
Copyright:Attribution Non-commercial


Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less





Understanding Navigability of Social Tagging Systems
Ed H. Chi, Todd Mytkowicz
Palo Alto Research Center3333 Coyote Hill Road, Palo Alto, CA 94304 USAechi@parc.com,
Given the rise in popularity of social tagging systems, itseems only natural to ask how efficient is the organicallyevolved vocabulary in describing any underlying documentobjects? Does this distributed process really provide a wayto circumnavigate the traditional categorization problemwith ontologies? We analyze a social tagging site, namelydel.icio.us, with information theory in order to evaluate theefficiency of this social tagging site for navigation toinformation sources. We show that over time, del.icio.us isbecoming harder and harder to navigate and provide anevaluation metric, namely entropy, that can be used toevaluate and drive system design choices.
Author Keywords
Social tagging, navigation, information access, ontologies,efficiency, evaluation, methodology, information theory,entropy.
ACM Classification Keywords
H.5.3 [Information Interfaces and Presentation]: Group andOrganization Interfaces—collaborative computing; H.1.2[Models and Principles]: User/Machine Systems—Humaninformation processing; K.4.3 [Computers and Society]:Organizational Impacts – Computer-supported collaborativework; H.5.2 [Information Interfaces and Presentation]: UserInterfaces
General Terms
: Design, Experimentation, Human Factors
The accumulation of human knowledge relies oninnovations in novel methods of organizing information.Subject indexes, ontologies, library catalogs, Deweydecimal systems are just a few examples of how curatorsand users of information environments have attempted toorganize knowledge. Recently, tagging has exploded as afad in information systems to categorize and clusterinformation objects [8]. Tagging has become a useful wayfor users to recall information sources for later use as wellas to communicate interesting nuggets of information toother users [11].The purpose of an organization scheme for any informationsystem is to help users browse, discover, and navigatethrough the information sources. A unique aspect of tagging systems is the freedom users have in choosing thevocabulary that can be used in tagging objects. First, tagsare not required to be placed in a hierarchy. Second, anyfree-form keyword using a combination of letters andnumbers is essentially allowed as a tag.Surprisingly, the use of free-form labeling may appear likea recipe for disaster but instead has turned into one of thenewest trends on the web. Indeed, social tagging has beenapplied to photos [6], videos [20], web pages [5], Wikipediaarticles, and academic paper citations [4], and in each of one of these cases, an organically-evolved vocabulary hasemerged to describe the content of the tagged objects.Some bloggers have written about the organic nature of theevolution of the tags in a social tagging system and how itcompares with ontologies, and perhaps the most well-known writing is Clay Shirky’s work [16].An important question is raised by Shirky’s essay, “Howand why do social tagging systems work?” This is enoughof a mystery such that a special panel during last year’sCHI2006 conference brought in experts to discuss this topic[8]. Some research has also emerged to explain andcharacterize the tagging phenomenon [10, 13]. Golder andHuberman studies the growth of tagging systems, andoffered a potential explanation for why objects acquirestable tag patterns [10]. Macgregor and McCulloch providean overview of the phenomenon and explore reasons whyboth social tagging as well as ontologies will have a placein the future of information access [13]. From a systemoriented perspective, Sen et al. studied how personaltendencies and community influences affect the way userstag items on a movie recommender website [15]. This setof initial research points to the fascination with socialtagging.During the CHI2006 panel, George Furnas mentioned that apotential cognitive process for explaining how socialtagging works might arise out of an analysis of the“vocabulary problem” [8]. Specifically, Furnas mentionedthat the process for generating a tag for an item that might
be needed later appears to be the same process that is usedto generate search keywords to retrieve a particular item ina search and retrieval engine [7]. He also noted thatgenerating a small set of keywords for an item is relativelyeasy for a single user, while generating a large set of keywords is a task best left for a group of users. In short,“different users use different terms to describe the samething” [8].Viewed from this perspective, social tagging and otherindexing systems are attempting to solve a mappingproblem. Social tagging, in a way, is trying to provide amap, which are summarizations of an explorable space.Using this map, users can navigate within the informationspace much more effectively than they otherwise would beable to do. The vocabulary problem in social tagging isthus asking the question of how groups of users effectivelygenerate this map (a set of keywords mapped to a set of items).Indeed, the raging debate between using ontologies vs.social tagging boils down to the fact that both approachesare trying to index the content for later retrieval. Thus, therelative efficiency in generating good maps could either restwithin predefined ontology controlled by a rule system, oremerge organically amongst the wisdom of a crowd.Therefore, our task in understanding how social tagging isevolving is reduced to the problem of understanding howsocial tagging affords social navigation. Given a tagvocabulary and its mapping to items, how effective is it indescribe that set of items?In this paper, we propose to use information theory toanalyze the effectiveness of social tagging. We haveapplied this methodology to the entire del.icio.us socialtagging systems. The results show that:1) The efficiency of social tagging is decreasing. First, theentropy of the tag vocabulary is increasing. Moreover, theentropy of documents conditional on tags,
, isalso increasing. Together, these two pieces of evidencemean that tags are becoming less and less descriptive.These results suggest that tag effectiveness is decreasing,but from a design perspective, there are ways to increasetag effectiveness.2) The efficiency of social navigation afforded by thetagging system is also decreasing. First, the entropy of thedocuments is increasing. Moreover, since
isincreasing, it is becoming harder for users to use the tagvocabulary to describe the document objects they want tobookmark. In other words, tags are becoming lessmeaningful in regards to providing salient navigability.3) We found that the entropy of user
continues toincrease, suggesting that there is a greater diversity of usersutilizing the social bookmarking system. The entropy of documents conditional on users,
, wasincreasing but has plateaued. Combining that with the factthat the total number of users is increasing, the datastrongly suggests that users are increasingly discoveringsimilar sources of documents, and that they overlap inplaces they visit more strongly than before.Finally, these analyses have direct design implications forsocial navigation tools. Not only do they provide amethodology whereby one can evaluate new system design,they also have strong implications for how people are usingthe web at large and provide motivation for new systems.
Social bookmarking systems have actually been a researcharea for some time, including systems such as Glance et al’sKnowledge Pump [9], Bouthors and Dedieu’s Pharos [2].However, these systems did not use social tagging. The useof free-form labeling to tag documents has actually beenaround for a while, but became a focus of the Web 2.0movement when Thomas Vander Wal described thephenomenon as “folksonomy” [18]. The phenomenon isnow used to describe Web-based technology for generatingfree-form labels that categorize contents collaboratively.The popularity of these social tagging systems perhaps canbe attributed to the benefits users perceive in easilyrecalling contents that might be useful later.The surprising aspect of the phenomenon is that, in contrastto professionally developed taxnomies, a folksonomyappears rather unsystematic, but over time, an order withinthe tags appears. Some have posited that this is due to thefact that social tagging systems dramatically lower the costof labeling items when compared to traditional taxonomies,because one does not have to be trained to use thetaxonomy [13]. Since many users can do labeling cheaplyand in a distributed fashion, many more objects are tagged.Moreover, since it does not use a controlled vocabulary, itcan easily respond to changes in the consensus of howthings should be classified. This is a point well-explored byShirky in his essay [16].There are just starting to be a handful of academic researchfocused on the dynamics of social tagging systems. Themost well-known so-far is probably Golder andHuberman’s initial work on understanding the usagepatterns of social tagging systems [10]. They characterizeda small subset of del.icio.us data and found that there ispotentially a growth pattern to the tagging system.Moreover, they found that, for an item, tags slowly stabilizeto a pattern in which the proportion of each tag is a fixedpercentage of the total frequency of all tags used.Library scientists have also started to look at social taggingand its relationship to traditional taxonomies. MacGreogorand McCulloch collected and discussed a set of argumentsrelated to the pros and cons of controlled vocabulary vs.free-form labeling [13].
 CSCW researchers have also started looking at the field as aarea for investigation. In particular, Sen et al. studied howtag selections are affected by community influences andpersonal tendencies [15]. They studied four different tagselection algorithms for displaying tags from other usersand found that user tagging behaviors changed dependingon the algorithm.Millen et al., on the other hand, introduced a systemdesigned for tagging intranet items in an enterprise [14].They collected some sample user data, and showed thatusers form social networks through patterns of their usageof others’ bookmarks.Within the HCI community, the CHI2006 panel organizedby Furnas et al. brought social tagging systems to theattention of HCI practitioners [8]. Furnas’ suggestion thatsocial tagging systems can be viewed from a “vocabularyproblem” [7] perspective directly inspired the approachused in this paper. More importantly, his comment pointedto the usefulness of social tagging systems as acommunication device that can bridge the gap betweendocument collections and users’ mental maps of thosecollections. Social navigation as enabled by social taggingsystems can be studied by how well the tags form avocabulary to describe the contents being tagged.MacGregor also refer to social tagging fundamentally as avocabulary problem in indexing: “terms assigned toresources that are exhaustive will result in high recall at theexpense of precision. Conversely, terms that aretoo specific will result in high precision, but lowerrecall.[13]” Sen et al. also referred to this as a fundamentalresearch problem in the conclusion of the paper: “thedensity of tag applications across objects may provideinformation about their value to users. A tag that is appliedto a very large proportion of items may be too general to beuseful, while a tag that is applied very few times may beuseless due to its obscurity.[15]”Indeed, to understand how tags have evolved for a largecorpus of tagged items such as del.icio.us, we need tounderstand whether the tags adequately describe the itemsbeing tagged. Moreover, we need a way to understand howthe social tagging system will evolve in the future. Will thetags lose their specificity?In short, our contribution is providing a methodology forunderstanding how tags in a social tagging system evolve asa vocabulary. And how well does social tagging affordsocial navigation of a large collection of sources? We areproposing the use of concepts in information theory toexamine these questions.
Information theory has long been applied to theunderstanding of human languages. Claude Shannoninvented much of information theory in 1948 to understandhow codes can be efficiently transmitted in acommunication channel. The concept of source coding andchannel coding in information theory is concerned with theefficiency and robustness of codes in communication.These concepts have been applied to human languages.Codes are essentially words, which should be efficient andshort if they are commonly used, e.g. “I”, “the”, “a”, “this”.Second, a language (that is, codes used in communication)should be robust to noise by having enough redundancy inthe system.The central idea in Shannon’s theory is the source-codingtheorem, which says that, on average, the number of bitsneeded to communicate the result of an uncertain event isgiven by its entropy. From a statistical perspective, asomewhat more intuitive understanding of 
is that itmeasures the amount of 
about a particular eventassociated with a probability distribution.The concept can be understood from a simple experiment.Consider a box containing many color balls from which weare drawing balls. If no single color predominates in thebox, then our uncertainty about the color of the ball ismaximal and the entropy is high. On the other hand, if thebox contains black colored balls more than other colors,then there is more certainty about the color of a drawn ball,and the entropy is lower. Intuitively, a gambler wouldprefer the second case, because it is possible to place betson black and win. In fact, in the extreme in which everyball is black, the entropy would be zero, and the gamblerwould win every time.Entropy thus measures the average amount of informationassociated with a drawn ball. Intuitively, in the third case,the color of the ball is a certainty, and there is noinformation conveyed by knowing the color of a drawn ball.In the first case, knowing the color of previously drawnballs tells a gambler a lot of information about how sheshould bet.Mathematically, given a discrete random variable
and itconsists of several events
, which occurs with probability
, the entropy
is given by:.Looking at the above equation, there are two basic ways inwhich entropy can change:(a) If the total number of events in
increases, entropy of 
 will increase. This is because entropy is defined as asummation of the values given by a function based on theprobabilities of 
. (Note that the negative log of aprobability is always a positive number.)(b) If distribution on
becomes more uniform, entropywill also increase.

You're Reading a Free Preview

/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->