Evaluation – Information Service

Library Knowledge Bulletin

This presentation will:
1. Introduce the information service 2. Present a number of methods for evaluating the information service 3. Note how these evaluation methods can be implemented

2

Knowledge Bulletin: Introduction
• A fortnightly ‘current awareness publication’ sent to all members of staff • Aims to maintain the professional currency of technical staff, including quantity surveyors, project managers and legal practitioners • Presents abstracts of articles from the past fortnight’s technical press, highlights new library items and new electronic resources • Includes hyperlinks to online resources with further information
3

Knowledge Bulletin: Development
• Originally written as a Word document, and sent in the body of an email • As part of corporate ‘re-branding’ exercise in 2009, began to be sent using an online email marketing tool (MailChimp)
4

MailChimp: Advantages for evaluation
• Provides ‘open reports’ – viewing who, and how many people, open each email • Allows link tracking – Showing how many people click on each individual link • Shows when, and how many times each, people open each email

5

Methodology
• Three sources of feedback: 1)Direct user feedback – Primarily in the form of users replying directly to the newsletter requesting documents that are not available electronically 2)Usage data from MailChimp 3)Brief survey questions included within the Knowledge Bulletin
6

Usage survey
• Possible methodology: 1)Inclusion of online survey such as Survey Monkey
– Pros: Many questions, professional appearance – Cons: Possibly blocked in Outlook, incompatibility with Blackberries

1)Email links
– Pros: Works in all systems, internal, allows users to write opinions in email (open-ended comments) – Cons: Possibly lower take-up (requires more thought)

7

Usage survey II: Option 1
• Survey Monkey:
– A survey through this service could either be included in the email (not recommended by MailChimp) or linked prominently in one or more issues of the newsletter. – Using the service’s templates and tools, this survey will be easy to set up and monitor.
8

Usage survey II: Option 2
• Email links
– Include a link in the email that creates a new email containing an open-ended survey to be sent to the library
9

Data analysis and presentation
• Graph usage data over time • Compare:
– Most popular sections – Open-rates based on time of day – Rates at which articles are requested via email vs clicked on if link is available

• Include written feedback • Results from survey
10

Necessary staff resources
• Tasks
– Collecting usage data from MailChimp (to be done on a regular basis?) – Reviewing email exchanges resulting from Knowledge Bulletins – Devising and monitoring brief user survey to include with Knowledge Bulletin

11

Decision making
• Ongoing process based on which links are popularly clicked on • Consider changing and re-ordering section titles based on which are popular/current • Take into account recommendations from readers • All changes can happen on a trial/temporary basis due to the nature of the publication

12