Competency N Angelo _ Roselle 1

Evaluate programs and services on specified criteria.

Introduction

Evaluating library programs, services, and products is vital to the overall success of a

library. Librarians use evaluation data to make decisions about the effectiveness of specific

products, to examine the usefulness of current programs, and to see the impact of new services.

Interestingly enough, competency for evaluation is related to competency for planning, research,

and management. Since evaluation is a systematic process, it requires accurate research about the

program/service, strategic planning to determine measureable goals and objectives for the

program/service, and managerial skills necessary for decision-making. Finally, in order to

conduct an evaluation of a service or program, librarians must move through an evaluation

process that includes, determining the specific criteria to assess, deciding on the methods and

measures to use, collecting data, and lastly, analyzing data.

Whether a librarian needs to assess the productivity of a new adult literacy program, or

the librarian wants to measure customer satisfaction of e-reference services, it is important to

evaluate based upon specific criteria. Although there are numerous types of evaluation criteria

that can be used, some librarians may evaluate the extensiveness of a service/program, efficiency

of resources, effectiveness of a service/program, service quality, impact that a service/program

has had on the library, and finally, the usefulness of a service/program (McClure, 2008, p. 183).

Once the criteria have been selected, the next step in the process is to decide which

methods and measures will be used for the evaluation. For instance, a common type of

evaluation measure used in library evaluation is output measures, which provides a quantitative

measurement of the output of a library service or program (e.g. Website usage statistics). In

addition, librarians may want to evaluate the performance or impact of a service/program, and
Competency N Angelo _ Roselle 2

there are measures that can be utilized for these evaluations as well; therefore, it is important to

understand the program/service evaluation needs from the outset of the evaluation process,

which will then help librarians determine the criteria and methods to use.

Data used for evaluations are broken down into two categories 1) quantitative and 2)

qualitative. For example, a librarian wanting to evaluate the usability or accessibility of a

library’s Website may want to gather quantitative data such as number of visits to the site, the

number of page views, the average time on the site, and the traffic sources. These are all

measurable data, resulting from the output of library service (i.e. the use of the Website). At the

same time, a librarian conducting a Website usability evaluation may also want to collect

qualitative data. For example, the librarian may want to observe library patrons accessing the

library’s Website in a natural setting, and then conduct follow-up interviews or focus group

sessions to get feedback about the service from the library patrons.

Both quantitative and qualitative evaluation methods provide librarians with data that can

be used to improve programs and services; however, there are also advantages and disadvantages

that must be recognized. An advantage of quantitative evaluation methods is that they have the

ability to provide measureable objective data, which can then be statistically analyzed; however,

numbers only tell part of the story, and quantitative evaluation methods lack insight into the

behaviors of users. On the other hand, qualitative evaluation methods give insight into users’

behaviors, but the downside is that the data collected are often subjective and not always

straightforward. In the end, if evaluation methods are efficiently used, librarians should have the

data needed to determine whether or not to continue, improve, or terminate specific library

services, programs, or products.

Competency N Angelo _ Roselle 3

Evidence

1) Webby Awards: JPL Cassini Mission Website (Library 251, Fall 2009)

My first piece of evidence is a Web usability critique of Jet Propulsion Laboratory’s

Cassini Mission Website, which won a Webby award in the category of science. Although this

assessment was not connected to a specific library program or service, I selected it as evidence

because it demonstrates my ability to evaluate a resource based upon specific criteria. The

specific criteria used for evaluation includes the usability and service quality of the site’s

content, structure and navigation, visual design, functionality, interactivity, and overall

experience. Library Websites are often a primary resource for library users, and librarians must

understand that a Website functions as a service for patrons; therefore, an evaluation of good

usability and accessibility must be systematically conducted to ensure user satisfaction. For these

reasons, I think the criteria I evaluated for the JPL’s Cassini Mission Website are also applicable

to any library that needs to assess its Website. In summary, it was through my evaluation of this

site that I learned about the importance of evaluating a library service from a user’s perspective,

the need to understand the usability issues that Web designers face, and seek out ways to

increase accessibility to library users.

2) Evaluation of brarydog.net (Library 283, Fall 2009)

This assignment required me to evaluate the Homework Help & Web Companion service

(brarydog.net) of the Public Library of Charlotte & Mecklenburg County. It was chosen as

evidence because it shows how a simple assessment of a library service can provide a library

with information necessary to help make informed decisions about a service. The goal for this

assignment was to determine the customer-centered organization characteristics of brarydog.net;

therefore, I evaluated specific characteristics including the library’s behavioral bottom line,
Competency N Angelo _ Roselle 4

reliance on research, understanding of the competition, and use of a full marketing mix. As I was

writing this assignment, I realized that many library programs and services are not customer-

centered, and as such, they are not providing ultimate customer satisfaction. Likewise, money,

time, and other library resources may be wasted if a library does not periodically evaluate the

effectiveness and usefulness of program or service. In the end, I learned that proper evaluation of

a library service requires that librarians understand users, competitors, and library resources.

3) Viva Las Vegas…7.5 reasons why I recommend The Librarian’s Brain (Library 246,

Spring 2009)

The final piece of evidence I present for this statement of competency is a short

evaluation I wrote about a library blog called The Librarian’s Brain. I chose to include this

assignment as evidence because the term “Library 2.0” has become a trending topic in librarian

circles, and many times it seems as if libraries want to use Library 2.0 technology for programs

and services without conducting a systematic evaluation of the technology; however, if a library

decides to start a library blog, use Twitter as a reference service, or even open a Flickr account to

show pictures of a tutoring program, it is critical for library management to understand Library

2.0 best practices. Also, it is the process of evaluating technology which enables libraries to

increase user satisfaction.

This evaluation demonstrates my understanding of blogging best practices. Specifically,

it shows that I have an ability to evaluate a blog service by looking at fundamental blogging

criteria including frequency of updates, usability, content, external links, and organization. As a

result of writing this assessment, I came to realize that even a seemingly easy to set up library

service (e.g. a library blog) ought to be effective, provide quality service to readers, and must

stand up to an evaluation by a knowledgeable librarian.

Competency N Angelo _ Roselle 5

Conclusion

This statement of competency demonstrates my abilities to evaluate library services and

programs based upon specific criteria. Knowing how to conduct methodical evaluations, assess

services and programs based upon specific criteria (e.g. usefulness, impact, usability,

accessibility, and efficiency), and offer suggestions for improving services and programs, are all

vital librarian skills. A question that is often asked of us LIS graduate students is how we can

transfer our skills to various work situations or environments; therefore, I respond to this

question by stating that the skills and knowledge I obtained to satisfy this competency will prove

useful in any information organization or future work setting. For example, my current internship

at JPL Library & Archives is providing me with opportunities to put my evaluation skills into

practice as I am evaluating digital image archival collections and seeking ways to improve

access. Furthermore, it is my understanding, experience, and coursework in evaluating services

that will allow me to evaluate a variety of library programs and services in the future, and make

suggestions for improvement. In closing, I feel confident that no matter the information center

that I find myself in later on down my career path, I will be able to continually demonstrate my

competency in evaluating library services and programs.

References

McClure, C. R. (2008). Learning and using evaluation: A Practical introduction. In K. Haycock

& B.E. Sheldon (Eds), The portable MLIS: Insights from the experts. (p. 179-192).
Westport, CT: Libraries Unlimited.