This action might not be possible to undo. Are you sure you want to continue?
0 tools Author/s & Organisation/s: Ellen Forsyth State Library of New South Wales Leanne Perry State Library of New South Wales Indicate which topic or stream you are submitting under Measuring our services Presenter/s name: Ellen Forsyth and Leanne Perry Postal address: State Library of New South Wales Macquarie Street Sydney NSW 2000 E-mail address: firstname.lastname@example.org email@example.com Phone: 02 9372 1525 02 9372 1523 Fax : 02 9273 1422 Abstract (200 words minimum to 350 words maximum) In 2008 the State Library of NSW developed the learning 2.0 program which was an online self-paced training program to allow public library staff across New South Wales to learn more about emerging technologies on the web. In 2010 we launched a follow up online program learning 2.1. With more than 1800 NSW public library staff participating in these programs we decided to explore how libraries plan and evaluate the success of using these web 2.0.tools in a library context. This paper will discuss the evaluation of the use of web 2.0 tools as well as providing background information on the purpose of evaluation. We will include research we have done into public library use of Flickr and the how public libraries are evaluating the effectiveness of this tool. Several public library working groups which the authors collaborate with are using web 2.0 evaluation tools in their work. We will use the Reference and Information Services Group as a case study and examine their use of Google Analytics on several wikis. We will discuss feedback on the evaluation information provided as part of the state wide learning 2.1 program.
As a result of working with a range of web 2.0 evaluation tools we will explore ideas for good practice, the importance of planning evaluation early in the process and discuss ideas about what the numbers actually mean. 1. Introduction In February 2010, the global average time spent per person on social networking sites was nearly five and half hours per month 1 which was an increase of more than 60% from February 2009. When considering specific countries, Italy with social network time per person just under six and a half hours per month (6:27:53) was the highest, however Australia is a close second with more than 6 hours per person per month (6:02:34) in terms of usage of social networking sites.2 Based on this data it not surprising a growing number of public libraries in NSW are using social networking sites or web 2.0 tools to connect and engage with their communities, their colleagues and their networks. The importance of understanding the impact of these social media tools on our business is as critical to libraries as it is to Julia Gillard, the RTA and QANTAS. This paper will explore what, when and how public libraries are or are not measuring the impact of using web 2.0 tools on their library business and why it is important that they do. 2. Background The New South Wales public library learning 2.0 program (learning 2.0)3 is a twelve week online self-paced training program to allow public library staff across New South Wales to learn more about emerging technologies on the web, with a focus on web 2.0 tools. In 2008 the State Library of New South Wales developed the learning 2.0 program for all public library staff in New South Wales. In October 2009 we launched a follow up online program learning 2.1. With more than 1800 NSW public library staff participating in these programs and many libraries using web 2.0 tools to provide library services and communicate within their library networks we undertook some research to explore how libraries plan and evaluate the success of using these web 2.0.tools in a library context. 3. Measuring the effectiveness of web 2.0 tools in NSW public libraries 3.1 Learning 2.0 program 3.1.2 Methodology In order to evaluate the longer term impact of the learning 2.0 program on individual participant skills and overall library services a nine month follow up survey was undertaken with participants who had completed the learning 2.0 program at least 9 months prior. The survey respondents were asked to comment on web 2.0 tools they are now using in their libraries and for what purpose were they being used, and what planning and evaluation strategies around web 2.0 tools were in place. For the purposes of this paper we have focussed on the results of the research which relate to planning and evaluation of web 2.0 tools.
In 2009 the 9 month follow up survey was sent to the 313 learning 2.0 participants who had completed the program. Seventy two of the 313 completing program participants responded to the 9 month follow up survey. Although this response rate is less 25%, 72 responses from a network of 99 public library services was considered a reasonable representation of initiatives and services across the network. 3.1.3 Key findings The follow up survey for the learning 2.0 program showed limited planning is undertaken for implementation of web 2.0 tools. Fifty percent of the libraries had not done any strategic planning about their use of web 2.0 tools, but consider options as they arise. This lack of planning may deliver good outcomes initially, but it can also threaten the sustainability of library services and communication offered via web 2.0 tools if inadequate planning delivers a shortfall of resources or competing priorities and tools. Planning can also play a key role in ensuring there is wide spread understanding within the organisation of where the web 2.0 tools fit in to the broader library strategy. It is too early to determine with any certainty the correct interpretation of these statistics. The lack of planning may indicate a lack of organisational recognition or support for web 2.0 initiatives, or it may mean that there is an informal environment where things ‘just happen”, or it may indicate that the library is in an experimental phase trialling tools for suitability and acceptance with target client groups. Even in a trial phase some planning, which does not have to be elaborate, time consuming or too structured, may yield better outcomes if it allows the organisation to utilise staff skills. The lack of planning also makes it unlikely that adequate evaluation has been planned, reducing capacity to prove the effectiveness of web 2.0 implementations in a consistent way. Despite a general awareness of the importance of evaluating services and tools generally to ensure quality and the most effective use of resources, 57.4% of libraries did not have any formal mechanism for evaluating their web 2.0 initiatives. This may highlight a trend to not evaluate web 2.0 tools with the same rigour as other library tools and services are evaluated. Alternatively, it may indicate that libraries are not using evaluation in a regular or systematic way across all aspects of their service delivery or that staff may lack awareness of tools that could make evaluation straightforward and meaningful. Interestingly, many web 2.0 tools have excellent built in statistics (such as Flickr pro accounts). There are also free web based analysis tools such as Google Analytics which are very useful for blog, wiki and website statistics. Many wikis also have useful built in statistics as well, so there is no shortage of available data. It is interesting that many people have identified evaluation strategies for their library’s use of web 2.0 tools. Without evaluation it is impossible to judge the effectiveness of the web 2.0 usage, and it limits the ability of staff to demonstrate the usefulness of the tool to the funding organisation. No library mentioned that they were evaluating or measuring web 2.0 based comments about their library or council. This can be easily done using Google alerts and is an excellent way to find out what people are saying about your organisation so that you can participate in the community discussion. Popular methods of evaluation amongst the libraries who did indicate they were evaluating their use of web 2.0 tools to provide services included the use of comments (29.6% of libraries), site visits (25.9% of libraries), client feedback (37%),
posts (24.1%) and views (24.1%). One point nine percent of libraries were using location of referrals amongst their evaluation tools. These methods could be easily used by the other libraries. This highlighted a need to provide more information to the NSW public library network about the options available to assist with the evaluation of web 2.0 tools. The rationale for undertaking an evaluation of the learning 2.0 program was twofold. Principally, we wanted to determine the effectiveness of web 2.0 tools for public libraries delivering services. However we also wanted to demonstrate the value of evaluating usage and outcomes. This research highlighted the need to undertake evaluation which informs key decisions about the management, maintenance and continuation of a program, service or product, as well as the need to encourage the evaluation of the use of web 2.0 tools in service delivery. The follow up learning 2.1 program included a core unit on evaluation and web 2.0 tools. This unit explored tools such as Google Alerts, Flickr Pro, Feedburner and Survey Monkey and encouraged libraries to consider evaluation as a critical element of service provision when using web 2.0 tools. 3.2 Flickr use and public libraries The learning 2.0 program generated high levels of interest by the NSW public library community about the use and impact of Flickr as a tool in the provision of library services. Globally Flickr is being used by an increasing number of public libraries for providing access to collections and services for their communities. Public Libraries are also using Flickr to build community engagement, however use of Flickr within NSW public libraries following the program remains limited. In 2009, to gain a better understanding of the use of Flickr in public libraries, we undertook research to identify public library expectations around Flickr as a tool, the strategic objectives of libraries using Flickr and how libraries had evaluated the impact of their presence on Flickr. 3.2.1 Methodology A survey was developed to examine the use, planning, evaluation and impact of Flickr in public libraries. Responses to the survey were requested from public libraries who had established a Flickr account. The survey was posted on a number of public library related e-mail lists (some based in individual countries and some international) and Flickr mail was used to request responses from individual libraries on Flickr. Dr Michael Stephens promoted the research on his blog4. One hundred and twenty nine libraries responded to the survey. Respondents were from a broad range of locations including Australia, United States, Canada, Europe, United Kingdom and New Zealand. All respondents were from public libraries serving populations from approximately 1700 to over 4 million people. 3.2.2 Key findings What are libraries using Flickr for? To understand the outcomes achieved by libraries using Flickr it is important to provide some context around what libraries are trying to achieve by using it. The two main uses by public libraries of Flickr were for recording events and promoting their collections, see Table 1.
Table 1. What does your library use Flickr for? Recording events Service promotion Promoting our collection Providing access to images from our collection Knowledge sharing Building knowledge and context of the images in your collection 69.6% 60.8% 35.2% 34.4% 28.8% 23.2%
Other uses specified included tours of libraries, providing photos to press and social media for promotional purposes, photography competitions, sharing information about library initiatives with other library professionals, work stalls at shows, advertising the library, building community, display of heritage images, and fun. One library indicated they were using the RSS feeds from their Flickr images to display on a video wall within their library whilst another library used Flickr to share progress on the development of their library building by uploading before, during and after photos of library building renovations. Recording events, promoting collections and services dominate the public library use of Flickr. These three linked ideas are all about helping people be more informed about the use of the public library, what is available in both collections and services and the recording of events (as both a promotion and a record). So if exposure of our collections and services are the most prominent objectives of using Flickr in libraries, have they been effective. An understanding of how many people we have reached and how our users used the images on Flickr are critical to measuring how successful the use of Flickr has been. Evaluating the impact of Flickr in public libraries Despite the importance of evaluating library services and tools generally to ensure quality and the most effective use of resources, evaluation of the impact of Flickr appears to be largely unplanned. Fifty six point seven percent of libraries did not have any formal evaluation methods. Figure 2. How do you evaluate the success of your Flickr account
Ho w do y o u e v a lua t e t he s ucc e s s o f y our Flickr a ccount ? 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% image/set views contacts client feedback referrals to council/library website from our Flickr account referrals to Flickr from council/library website we don't have any formal evaluation mechanisms 12.7% 11.0% 11.9% 44.1% 32.2% 57.6%
This is consistent with a general trend in NSW public libraries not to evaluate web 2.0 initiatives. Fifty seven point four percent of libraries did not have any formal mechanism for evaluating their web 2.0 initiatives.5 When libraries did undertake formal evaluation, the impact of Flickr was largely measured by the number of page views for their Flickr account. Forty four percent of respondents indicated this was how they evaluated the impact of the library’s Flickr account. This is not surprising as Flickr Pro accounts have excellent built in which are able to provide reasonably extensive data on visits and usage of Flickr accounts. This access to data means that critical evaluation data is available to Flickr account owners at a nominal cost and requires no extra resources for compilation and basic analysis. Although 44% of libraries use this data for their evaluation of Flickr as a tool, given the ease of access and low level of resources to quite significant data sets it would be reasonable to conclude libraries still remain relatively unaware of the supplementary services and information available as part of a Flickr Pro account. Most libraries were unable to provide statistics on visitors to their Flickr site. Only 22 libraries from 129 libraries were able to provide usage data. Only 14 libraries indicated an awareness of the level of referral from their Flickr site to their library website. Many libraries were unaware of the statistics which could be provided by Flickr as part of the Pro account. More than 32.2 % of libraries indicated client feedback as an evaluation strategy although it is unclear how this feedback was used and what level of compilation of information and analysis was undertaken, what systems were in place to respond to feedback and whether this feedback resulted in changes. There appears to be little correlation between the type of evaluation mechanisms used and factors such as the length of time a Flickr account has existed, who manages the Flickr account or what the library is using Flickr for. However, although not surprising, it is worth noting that there is a clear correlation between the level of planning for ongoing use and the levels of formal evaluation undertaken (see figure 3). When asked about how the libraries planned their ongoing use of Flickr, 54.6% said that they considered options as they go. Six point seven percent had a Flickr strategy included in their strategic plan, 9.2% of libraries has a policy about their use of Flickr, 21% had protocols about adding images to Flickr, and 26.1% were developing policies, protocols or strategies for their use of Flickr. 17.6% did not plan. These statistics show a low level of planning for the use of Flickr with over half of the respondents not planning ahead, but considering options as they arise. The respondents were not asked how this compared to their general planning for new services for their library. It is feasible to speculate that other areas of library services receive more planning than the involvement on Flickr, however this has not been specifically addressed in the research. Where libraries plan or already have a policy, where there are protocols in place for using Flickr or if Flickr is included in their strategic planning processes; formal mechanisms for evaluation tend to exist. If a library has not planned for ongoing use or are considering options as they go than they are much less likely to have formal evaluation mechanisms in place.
Figure 3. Evaluation and planning ongoing use of Flickr
Ev a lu a t io n a n d p la n n ing o n g o in g u s e o f Flic kr ? 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% W e have a Our Flickr W e have policy straetgy is protocols about our included in for adding use of our images to Flickr strategic our Flickr plan account W e are W e just We do not currently consider plan developing options as policies, we go protocols and/or strategies for our
image/set views contacts client feedback referrals to Flickr from council/library website we don't have any formal evaluation mechanisms
Low levels evaluation indicate a lack of confidence about how Flickr fits into current library service provision as well as concerns about the longevity, reach, effectiveness of Flickr as a tools for use in libraries. Without some investment in terms of planned evaluation, which include clear performance indicators and specified measures of success, the outcomes, benefits and limitations of Flickr for library services will continue to be unclear. It will remain impossible to judge the effectiveness of using Flickr to provide access to library services or collections and so limit the ability of libraries to demonstrate the usefulness of the tool to the funding organisation. The results of this research indicated that libraries are cautious in their approach to their use of Flickr and consequently resources committed to planning, evaluation and promotion of Flickr content is minimal. However, ironically, a low emphasis on evaluation of use of Flickr as a tool means that it is difficult to assess widely what the outcomes for public libraries using Flickr are. 3.3 Using web 2.0 tools to build the NSW public library network Over 2,300 people work in public libraries in New South Wales. There are State Wide special interest groups within this public library community of marketing, multicultural, evaluation, home library service, document delivery, young people, and reference. As these are genuinely state wide groups, online communication is really important as people are not always going to have the opportunity to meet together. For most of the groups there is an annual seminar, and there are face to face meetings, which only some of the interested staff are able to attend. The state wide groups of New South Wales public library specialist staff use e-mail lists, wikis, blogs, forums and twitter as part of their communication with each other. Google docs is also used by steering committees and other small groups as a planning and collaboration tool. The knowledge gained by people during the New South Wales Public Library learning 2.0 program6 helps bring about this usage as people learned about web 2.0 tools and started using them as part of the program This course gave people skills in a range of web 2.0 tools and these skills were put to use for state wide collaboration7.
Some of the groups are analysing their group use of specific web 2.0 tools as part of the measure for effectiveness of communicating this way. We will be talking about the work we have been doing with some of them and our roles in the evaluation work as well. 3.3.1 Reference and Information Services Group and Google Analytics We are using web 2.0 tools to evaluate use of other web 2.0 tools. Google Analytics have been set up for the wikis used by the reference and information services group. This comprises three wikis: • Reference excellence wiki89 • Readers advisory wiki10 • Reference and information services group wiki11 The Reference excellence wiki provides online, self paced training for New South Wales public library reference and information services staff. It teaches staff about reference work. It is based on the Ohio Reference Excellence program 12, however new modules are being added by local staff. The Readers advisory wiki is where this group stores seminar information, meeting agendas and minutes, book lists, reading challenges and much other information of relevance for New South Wales public library staff providing readers advisory services. The Reference and information services group wiki contains seminar information, meeting agendas and minutes and other information of relevance for New South Wales public library staff providing reference and information services. Each of these wikis has a very different pattern of use, and the patterns remain fairly stable over time. Much of the information about this has been learned using Google Analytics on each of these sites. The Readers Advisory wiki and the Reference and Information Services Group wiki can also provide some additional information about the people who are members of the wikis and how they are participating as members or as writers through the information provided by Wetpaint which is the software both of these wikis use. The information from Google Analytics which has been used to analyse the use of these wikis should be regarded as indicative information and the big picture patterns looked at, rather than a too detailed analysis. It is a very useful tool for seeing trends and patterns in use. The Reference excellence wiki, which uses Media wiki as its software, has a very different usage pattern to each of the other wikis. This information has been obtained through Google Analytics which has been providing information about this wiki since its start. People tend to spend between 8 and 12 minutes on this site, which looks like people are actually using the training modules. The use of wiki is predominantly from New South Wales, although there are visits from other states and territories. About 50% of visitors are repeat visitors, which again seems to indicate that people are working their way through training modules. There is a bounce rate of around 30% which means that most people have found the site they really wanted to look at. This is reinforced as about 45% of people make direct access to this site, and about 37% find this site through a search engine. As well as using Google Analytics for part of the evaluation of the Reference Excellence wiki there are surveys which provide qualitative data as well.
The most important area is not the comparisons between the wikis, but rather the comparison within each wiki about use over time as the example from Ref-ex shows. Ref-ex October 2009 November 2009 December 2009 January 2010 February 2010 March 2010 April 2010 May 2010 June 2010 July 2010 August 2010 September 2010 Time on site 9.30 minutes 7.30 minutes 9.00 minutes 9.00 minutes 8.00 minutes 12:30 minutes 11 minutes 7.30 minutes 7 minutes 17.30 minutes 9.00 minutes 7.30 minutes % new visitors 54% 51% 50% 42% 50% 43% 51% 51% 51% 61% 56% 53% Bounce rate 36% 37% 37% 34% 32% 35% 37% 31% 37% 32% 27% 40%
While there are variations from month to month, they are all in much the same range except for the much longer time in July.
This data visualisation from Manyeyes13 is another way to explore the data.
Manyeyes is a free data visualisation tool, which may help you in exploring the meaning of your evaluation, or in conveying meaning to other people about your data. In 2007 the British Library included a count of the number of blog posts14 which were about their exhibition, Sacred, as part of their evaluation of that exhibition. As well as a total count of blog posts, over 1200, they included extracts from, and links to, some of the posts. It is important that the blog posts are included in the evaluation, even if it is not clear in detail what is actually indicated by 1200 blog posts about one exhibition other than it was an exhibition which people wanted to talk about and share their experiences. It is a good idea to set up a Google alert, or series of alerts for your organisation and any key events or exhibitions as this is a very easy way to monitor comments. 3.3.2 Seminars and twitter use Several seminars of relevance to public libraries, co-organised by New South Wales public library staff are run at the State Library of New South Wales each year. When seminars are being advertised the hashtag for the day is also included in the publicity. On the day of the seminar people present are reminded of the hashtag, and and an e-mail is sent to the relevant New South Wales public library lists on the day with information about how to follow the seminar by twitter. Twitter was also covered in the learning 2.1 program15 provided to New South Wales public libraries. The number of people tweeting varies from event to event, but each time there are tweeted and e-mailed comments about the value of having a twitter feed for an event that someone can’t actually get to. We realise many other events, such as Information Online are using twitter feeds, but we are actively promoting very
targeted one for the New South Wales public library network and comments about this are used as part of our evaluation of events. For example for the rugame seminar which ran over one day at the State Library and half a day online, there were 540 tweets with 51 people tweeting16, for the annual Reference and Information Services Group seminar in 2010, there were 268 tweets 25 people tweeted and not all of them attended the seminar 17. Geographic information can be available as well, in more recent collections. Twitter featured as a communication for the 2010 Reference and Information Services Group seminar. One of the co-organisers was tweeting information about the seminar encouraging bookings. This helped to publicise the hashtag for the day. Several of the people tweeting were not at the seminar and were tweeting about not being there or commenting on tweets from the seminar. Since the seminar there has also been tweeting about how people enjoyed the day and what they have been doing with their information. The comments received on Twitter are evaluated in a similar way to comments received as part of the formal evaluation for each seminar. We are still experimenting with how to evaluate the use of twitter, but are reporting on the use as part of an overall evaluation strategy, particularly favourable comments about the events. As the role and use of Twitter as a tool for engaging with your community continues to grow it is worthwhile considering other tools which may support analysis of its effectiveness. We will highlight some possible tools to consider for use in analysing your organisation’s twitter account, or for looking at comments made about your organisation. This is just to show what is possible with a series of free web 2.0 tools. It is important to keep in mind that these are indicative only, and helpful for looking at big picture patterns rather than detail. They also can be reported as part of a broader evaluation rather than in isolation. Tweetstats18 allows to you see patterns in your twitter account. This may be helpful if your organisation is seeking to explore the patterns of twitter use within the organisation. You can look up anyone’s twitter handle this way to see patterns so you could contrast your organisation with a similar one. It also shows who you most reply to and retweet. For example have a look at the Tweetstat for the National Library of New Zealand 19 and the National Library of Australia 20 to see different patterns of use. Tweet effect21 is about the gaining and losing of followers. You can plot what tweets may have caused the change in followers. This may be useful for looking at patterns of new followers, and also what tweets may cause the loss of followers. Tweet reach22 explores how many people may (and the emphasis is on may) read a hashtag. The reach is calculated by the number of followers. There are various tools to help measure how many people are using twitter as a way of talking with your organisation, you can use the new twitter layout to look for mentions of your organisational twitter handle (need to be logged in to do this), or you can use Twitiq23 (which you do not have to log in to use). Favstar24 allows you to search for who has favourited your tweets. This could be used to help see interest, reach and influence. This is not a figure to use in isolation. Have a look at who has favourited tweets by Cory Doctorow25.
Are people retweeting you, and do you want this separate from mentions of your twitter handle, then you can use Retweetrank26 which is searchable by twitter handle. Social mention27, like Google alerts28, is a broad spectrum tool for finding mentions of your organisation in social media. Google alerts will also search social media as well. It is helpful to have alerts set up for variations on the name of your organisation to not only track comments being made about your organisation, but to explore what is actually being said about your organisation, favourable and possibly unfavourable. You can also use the results from the alerts in reporting, just as the British Library reported blog postings about their exhibition, Sacred. 4. Conclusion Although the use of web 2.0 for service provision in NSW public libraries is increasing it is clear that the take up, usage and anticipated outcomes of web 2.0 tools for library service provision is still relatively sporadic and experimental. Low levels of planning and evaluation indicate a lack of confidence about the how these tools fit into current library service provision as well as concerns about the longevity, reach, effectiveness of these tools. Consequently it appears minimal resources have been allocated to the strategic implementation of web 2.0 tools in public libraries. Without some significant investment in terms of time, planning and evaluation the outcomes, benefits and limitations of web 2.0 tools will continue to be unclear. Using web 2.0 tools within a professional networking context appears to be more likely to include a strategy around the evaluation of these tools. While it is likely that this demonstrates an increasing awareness over time of measuring the effectiveness of web 2.0 tools, it also indicates the importance of an environment which encourages experimentation with both the use and evaluation of web 2.0 tools. The principles of planning, implementing and managing a new service or project within a library context are the same regardless of the mechanism being used to deliver the service. As with any other service, evaluation is critical to the ongoing viability and sustainability of a service using web 2.0 tools. Clear articulation of the objectives for using the web 2.0 tool, identification of the measures of success and an understanding of the tools available to measure success are as relevant with web 2.0 tools as they are when evaluating the success of any program. There are many measurement tools available to use with web 2.0 tools which are simple to use and extrapolate indicative data. It is important to note that that these tools should be part of the overall evaluation strategy rather than the sole tool for evaluation. Interestingly the availability of web 2.0 evaluation tools means that measuring the effectiveness of web 2.0 in service provision does not have be an onerous and heavily resourced task. As with any evaluation the measuring of success of services provided using web 2.0 tools should be in proportion to the service being delivered and the audience it is trying to reach.
The Neilsen Company neilsenwire http://blog.nielsen.com/nielsenwire [accessed 11 October 2010] The Neilsen Company neilsenwire http://blog.nielsen.com/nielsenwire [accessed 11 October 2010] 3 NSW Public Libraries Learning 2.0 http://nswpubliclibrarieslearning2.blogspot.com/ [Accessed 17 September 2008] 4 TTW mailbox: Flickr research down under Tame the web 12 February 2009 http://tametheweb.com/2009/02/12/ttwmailbox-flickr-research-down-under/ [accessed 14 October 2010] 5 Forsyth, E; Joseph, M; Perry, L Tooling up for web 2.0: together, alone , ALIA 2009 National Library and Information Technicians Conference http://conferences.alia.org.au/libtec2009/Documents%20for%20Links/Perry,Joseph,ForsythALIA09.pdf [14 October 2010] 6 New South Wales Public Library learning 2.0 http://nswpubliclibrarieslearning2.blogspot.com/ [accessed 6 October 2010] 7 Networking: State Library of New South Wales http://www.sl.nsw.gov.au/services/public_libraries/networking/index.html [accessed 6 October 2010] 8 Reference excellence wiki http://wiki.libraries.nsw.gov.au/index.php/Reference_excellence [accessed 28 September 2010] 9 For more information about this see Balharrie, Ross, Forsyth, Ellen and Johnston, Cathy Reference excellence from US to Oz., Information Online 2009 http://tiny.cc/vc6an [accessed 28 September 2010] 10 Readers advisory wiki http://readersadvisory.wetpaint.com/ [accessed 28 September 2010] 11 Reference and information services group wiki http://referenceandinformationservices.wetpaint.com/ [accessed 28 September 2010] 12 Ohio Reference Excellence http://www.olc.org/ore/ [accessed 7 October 2010] 13 Manyeyes visualizations Time on site and new visitors [ref=ex] http://manyeyes.alphaworks.ibm.com/manyeyes/visualizations/time-on-site-and-new-visitors [accessed 11 October 2010] 14 British Library, Sacred : blog cuttings http://www.bl.uk/onlinegallery/features/sacred/blogcuttings.html [accessed 29 September 2010] 15 New South Wales public library learning 2.1 http://newsouthwalespubliclibrarylearning21.blogspot.com/ [accessed 29 September 2010] 16 ru game on Twapper keeper http://www.twapperkeeper.com/hashtag/rugame2010 [accessed 11 October 2010] 17 Sumarizr Twapper Keeper archive hashtag risg2010 http://summarizr.labs.eduserv.org.uk/?hashtag=risg2010 [accessed 11 October 2010] 18 Tweetstats http://tweetstats.com/ [accessed 7 October 2010] 19 Tweetstats for National Library of New Zealand http://tweetstats.com/graphs/nlnz [accessed 7 October 2010] 20 Tweetstats for National Library of Australia http://tweetstats.com/graphs/nlagovau [accessed 7 October 2010] 21 Tweet effect http://tweeteffect.com/index.php [accessed 7 October 2010] 22 Tweet reach http://tweetreach.com/ [accessed 7 October 2010] 23 Twitiq http://twitiq.com/ [accessed 7 October 2010] 24 Favstat http://favstar.fm/ [accessed 7 October 2010] 25 Favstar, tweets by Cory Doctorow http://favstar.fm/users/doctorow [accessed 7 October 2010] 26 Retweetrank http://www.retweetrank.com/ [accessed 7 October 2010] 27 Social mention http://socialmention.com/ [accessed 7 October 2010] 28 Google alerts http://www.google.com/alerts?hl=en [accessed 7 October 2010]
This action might not be possible to undo. Are you sure you want to continue?
We've moved you to where you read on your other device.
Get the full title to continue reading from where you left off, or restart the preview.