Evaluation – Information Service

Library Knowledge Bulletin

This presentation will:
1. Introduce the information service 2. Present a number of methods for evaluating the information service 3. Note how these evaluation methods can be implemented


Knowledge Bulletin: Introduction
• A fortnightly ‘current awareness publication’ sent to all members of staff • Aims to maintain the professional currency of technical staff, including quantity surveyors, project managers and legal practitioners • Presents abstracts of articles from the past fortnight’s technical press, highlights new library items and new electronic resources • Includes hyperlinks to online resources with further information

Knowledge Bulletin: Development
• Originally written as a Word document, and sent in the body of an email • As part of corporate ‘re-branding’ exercise in 2009, began to be sent using an online email marketing tool (MailChimp)

MailChimp: Advantages for evaluation
• Provides ‘open reports’ – viewing who, and how many people, open each email • Allows link tracking – Showing how many people click on each individual link • Shows when, and how many times each, people open each email


• Three sources of feedback: 1)Direct user feedback – Primarily in the form of users replying directly to the newsletter requesting documents that are not available electronically 2)Usage data from MailChimp 3)Brief survey questions included within the Knowledge Bulletin

Usage survey
• Possible methodology: 1)Inclusion of online survey such as Survey Monkey
– Pros: Many questions, professional appearance – Cons: Possibly blocked in Outlook, incompatibility with Blackberries

1)Email links
– Pros: Works in all systems, internal, allows users to write opinions in email (open-ended comments) – Cons: Possibly lower take-up (requires more thought)


Usage survey II: Option 1
• Survey Monkey:
– A survey through this service could either be included in the email (not recommended by MailChimp) or linked prominently in one or more issues of the newsletter. – Using the service’s templates and tools, this survey will be easy to set up and monitor.

Usage survey II: Option 2
• Email links
– Include a link in the email that creates a new email containing an open-ended survey to be sent to the library

Data analysis and presentation
• Graph usage data over time • Compare:
– Most popular sections – Open-rates based on time of day – Rates at which articles are requested via email vs clicked on if link is available

• Include written feedback • Results from survey

Necessary staff resources
• Tasks
– Collecting usage data from MailChimp (to be done on a regular basis?) – Reviewing email exchanges resulting from Knowledge Bulletins – Devising and monitoring brief user survey to include with Knowledge Bulletin


Decision making
• Ongoing process based on which links are popularly clicked on • Consider changing and re-ordering section titles based on which are popular/current • Take into account recommendations from readers • All changes can happen on a trial/temporary basis due to the nature of the publication