A Usability Analysis of the WorldCat Discovery Beta

Interface for J. Murrey Atkins Library

Submitted by:
Kate Calabro
Tayler Green
Rabee Khan
ENGL 4181, Writing User Documents
24 February, 2016
Submitted to:
Dr. Gregory Wickliff, Assoc. Professor of English
Rachael Winterling, Usability Coordinator

Executive Summary
For our ENGL 4181 Writing User Documents class, we were tasked with analyzing the usability of the WorldCat
Discovery beta interface. Because the library plans to implement WorldCat Discovery in place of WorldCat Local,
these usability tests were developed both in hopes of challenging problems from the old interface and the sight that
they would reveal additional problems with the Discovery interface. We chose 5 users in an attempt to represent a
diverse range of undergraduate students, with users being both male/female, native and international students,
freshman to senior levels of study, and including varying
majors among the mix.
The usability tests were performed using our groups laptops,
a printed task list, and users were observed by a group
member either recording or taking notes. The users had very
few problems performing the tasks required of them on the
interface, and all of the participants finished the tasks in
under 10 minutes. Interestingly enough, since our users
ranged from novice to advanced experience with the previous
interface WorldCat Local, the tasks on the Discovery
interface seemed to be an equal difficulty level for all
participants, and both the problems encountered and comments received were similar. The most notable difficulties
were able to be broken down into two main groups of problems:
1.

Layout and Design

inability to locate the sort by function
inability to find the due date of library materials
distractions from the sidebar location of other relevant searches

Much of the layout and design complaints stemmed from the size of the text on the interface, which therefore
interfered with the ability to find certain features, such as the sort by function and due date of library materials.
One user had a notable complaint that the sidebar that appears after clicking on a search result was very distracting,
which led our group to consider the option to have a collapsible or minimized sidebar, and therefore the possibility
of having a user personalized site.
2.

Content, Language and the User

general user apprehension to use site, test features
lack of user knowledge of key library search terms
downloading articles that would lead to the full content and not simply an abstract

The usability testing uncovered as much about the interface as it did the J. Murrey Atkins Library users. Two out of
the five users had no experience with the original WorldCat Local site, and out of those two users, one claimed she,
...never tried to use it because it looked scary. When considering some of the verbal feedback we received from
users who had trouble deciphering the language on the site (i.e. the definition of a peer-reviewed article) or users
that grew frustrated with finding and opening articles, which overall interfered with their navigation of the site, the
group felt that some form of a tutorial would be a necessary addition to the new WorldCat Discovery.

ii

Table of Contents
Title Page

ii

Executive Summary

iii

Table of Contents

iv

Introduction

Methodology

iv

Test Results
1. Layout and Design Problems
2. Content, Language, and the User Problems
Supplemental Qualitative Results
Visualizing Quantitative Results

vi
vi
vii
viii
viii

Findings and Recommendations

ix

References

xii

List of Figures
Figure 1.1: User Demographics
Figure 1.2: Usability Test Task List
Figure 1.3: User Post-Test Questions
Figure 2.1: WorldCat Local versus WorldCat Discovery
Figure 2.2: Distracting sidebar
Figure 2.3: Example A. and B. Search Functions and Text Size
Figure 2.4: Example C. Refining the Search
Figure 3.1: User Quotes by Task
Figure 4.1: Suggested WorldCat Discovery Search Visual
Figure 4.2: Suggested WorldCat Discovery Short-Term Changes

iv
iv
v
vi
vi
vii
viii
viii
x
x

List of Tables
Table 1.1: Common User Problems by Percentage

ix

ii

A Usability Analysis of the WorldCat Discovery Beta

Interface for J. Murrey Atkins Library
Introduction
With WorldCat Local having been introduced to the University of North Carolina at Charlotte in 2013, various
usability tests have been done since then to identify, make recommendations, and solve problems associated with the
interface. The J. Murrey Atkins Library will be introducing the latest interface from OCLC, WorldCat Discovery.
The primary purpose of the usability tests performed on the library discovery interface, Beta version, were to find
areas of frustration and inconsistency for users within the interface.
The J. Murrey Atkins library got in touch with Dr. Wickliff in the English department to help conduct usability tests.
Each group was assigned the task of creating usability tests for the beta site, along with finding participants to take
the tests. Before starting our project, our class met with the usability coordinator, Rachael Winterling, to ensure an
understanding of the usability testing and basic library interface system in place. Our team, which consisted of three
people, devised a task list for usability testing that explored various, popular features of the interface. The group
then recruited a variety of participants for this test to meet the wide demographic of undergraduate users at UNCC.
This ensured an equal representation and diverse results for the usability tests.
The scope of this report will address the two main issues with the WorldCat Discovery beta interface, being the
Layout and Design problems, and the Content, Language and User problems. The goal of the results from the
usability tests will relay the information collected to the librarians at UNCC, as well as designers at OCLC to
improve the new WorldCat Discovery interface.

iii

Methodology
In an attempt to replicate the wide variety of target users the J. Murrey Atkins Library serves, our group recruited
participants from different majors, levels of study, gender, and expertise with the library interface. Our group
recruited participants by word of mouth. Each member of the group asked people from their social circles. This
included friends, roommates, etc.
Before the start of the test, the participants were asked a series of pre-questions to create a demographic of the
library interface users, in Figure 1.1 below:

Figure 1.1: User Demographics

To perform the usability test, the group

provided the user with a task sheet.
After a sample user test on WorldCat
Local, the group constructed a task list
that explored several of the interface
features, in the hope that the tasks would
reveal usability problems with WorldCat
Discovery. When the facilitator either
began the recording or the notetaker was
ready, the test was administered to the
users. The test took about 10-15 minutes
depending on the user. The task list is in
Figure 1.2, to the right:
Figure 1.2: Usability Test Task List

The group provided participants with a laptop, cleared of history and cookies for each test. One of the five usability
tests were conducted using laptops installed with the OBS Studio software, which captured both audio, and video of
the webcam and screen captured the browser. The other four usability tests were supplemented with a note taker to
record observations on body language, signs of frustration, and where and how the user had problems with the tasks.
Facilitators and notetakers encouraged the users to think out loud through their execution of the tasks, and reminded
users that the test was on the library interface, and not of the user, himself. The group did not provide assistance to
the users when they had questions about the site, and only spoke if a user took an exceptionally long amount of time
or expressed great frustration with a task to move on to the next task, keeping the user test between the average 1015 minutes.

iv

Following the test, the users were asked a series of post-test questions. These questions helped give a better picture
of what the users thought of the interface. The users were also given the opportunity to provide any other additional
feedback the test and questions may not have covered, in Figure 1.3, below:

Figure 1.3: User Post-Test Questions

Our user participants all completed the usability tasks and questions. Our findings of these usability tests were
constructed using charts, tables, screenshots, and other visual aids from the data gathered by note-taking and
reviewing the two user test videos.

Test Results
While four out of five users were able to complete all of the tasks, the users did encounter difficulties as they
attempted to complete the tasks. It was noted that the problems that the users encountered were mostly the same
among all the users. The group was able to divide the main problems into two categories, which further delve into
their respective solutions.
1.) Layout and Design Problems
There could be a potential bias of the usability test results regarding layout and design, considering that because two
out of the five users had never used the library interface, the remaining three had experience with the WorldCat
Local layout, and had preferences for the initial layout. During the post-test questions, when asked what changes
could be made to the interface, one user suggested the the WorldCat Discovery search mimic the old WorldCat
Local layout, in terms of having the tabs present to sort the search. The old and new search engines are below, in
Figure 2.1:

Figure 2.1: WorldCat Local versus WorldCat Discovery

It was brought to the groups attention by one user who admitted she never used the library site in apprehension of
having no idea how to use it, that the layout of WorldCat to a novice user may not appear to be user friendly. The
WorldCat Discovery search offers no visible help or tutorial function on either the initial search page or the search
results page.

Figure 2.2: Distracting sidebar

vi

One complaint our group received from the a user was that after clicking on a search result and directed onto the
next page, that the sidebar with other relevant searches to the left side of the screen was overwhelming to the page
[outlined in red in Figure 2.1 above] , or in the users words, This is so distracting. Is it for what I clicked on, or
what? The user then realized that to find the information required of the tasks, she would use the drop down bars on
the right instead.
The interface lacks the ability to collapse that side search bar. Perhaps, like the right side of the page, where the
information is not revealed without being clicked on, the relevant searches on the left side of the page could be a
drop-down option. Otherwise, the width of the left sidebar with relevant searches should be able to be collapsed or
minimized to a degree which would not distract from the source that the user clicked on.
2.) Content, Language, and the User Problems

Figure 2.3: Example A. and B. Search Functions and Text Size

When users were asked in Task 1 to add the first search result to their feature My List, we had five out of five
users complain that the My List function was difficult to find due to the size of the text. In turn, users did not
know the function of either of the two buttons in Example B., or that the second button was the Add to My List
function. Most users seemed hesitant to click on the buttons without knowing their function.

vii

Figure 2.4: Example C. Refining the Search

The tasks regarding changing the formats of the search, like sorting for an audiobook or having only peer-reviewed
articles come up revealed much about the librarys users and the user-friendliness of the library interface. When
users were asked to sort articles by peer-reviewed results only, one user expressed her frustration with formats
available to her, as, I dont even know what the difference between a peer-reviewed article and normal article is.
With all five users complaining about the size of the text when searching for the function, and two of the five
thinking out loud that they did not know the function of the My List button, it is clear to our group that the lack of
explanation on of the sites features poses a problem for users navigation and understanding of the site and the
features available to them.
Supplemental Qualitative Results:
Our users were encouraged to think out loud while navigating the beta interface to that we could get a better idea of
what they were thinking. Some of our users became distraught, while more of them were unsure if they found the
right feature we were looking for. Here are some of the direct quotes with what the users said and how they went
through some of the tests:

Figure 3.1: User Quotes by Task

Visualizing Quantitative Results:

viii

As mentioned above majority of the problems that the users encountered were similar throughout all the user tests.
Some of the most common problems included users having trouble opening articles on external links, where users
were not always able to find the due dates for material that could be checked out of the library, the inability to
collapse the search bar, the inability to locate the sort feature to sort by newest to oldest, and finally, the one
problem that all users had in common was, the complaint that the text size was really small. Below is a breakdown
of the percentage of how many users experienced each major problem we had.

Table 1.1: Common User Problems by Percentage

Findings and Recommendations

The findings from these five user tests helped us realize that the new system is a lot more efficient than the old one,
but there are some areas that could still improved. Even though the majority of our users completed the user tests
with only minor concerns and problems, all of the users had suggestions for improvement, which created a
productive, collaborative dialogue for how the user experience could be improved in future updates of the WorldCat
Discovery interface.
Taking both the usability tests and the conversations from the users into consideration, the group has produced a few
suggestions for change, and has a drawn a mock-interface with those suggested changes. For the short-term changes,
the group felt that the most prevalent issue with the interface and the easiest fix would be to increase the font size of
the interface. By doing so, the problems with finding the due date of library materials and the drop down bars to sort
searches would be fixed, in turn.
To appease user preference and thoughts on user-friendliness of the site, and the suggestion of one of the users we
tested, the group also thinks that a potential solution to the design of the new interface would be to use the WorldCat
Local search layout of the initial search. The Local search interface provided tabs that would specify the search
down to the format desired (i.e. articles, books, etc.). The Discovery search currently requires the user to do a
general search, then find the appropriate check mark on the sidebar to narrow the results. The ability to narrow the
searches down immediately would save the user time and that extra step of scrolling through the options on the
following page of results.

ix

Figure 4.1: Suggested WorldCat Discovery Search Visual

Because of our own usability tests, which revealed that some users did not understand the language of searching
academic sites, the final quick-fix to the site before its actual release would be to provide definitions of the unclear
terms and functions on the site. This could be accomplished by highlighting the terms, and as users hover their
mouse over the term or function, a short definition or explanation could appear. In our example below, if the user
were to hover over a peer-reviewed article to refine their results, the definition of the term would appear, or if they
did not know what the My-List function was for, the user could hover their mouse of the button and receive a
result without having to click away to another page.

Figure 4.2: Suggested WorldCat Discovery Short-Term Changes

As for long term changes, the group would like to see the design of the interface be adaptable to user preference, so
that if a user finds the left sidebar present in searches distracting, they should be able to collapse or minimize the
width of the column. With the idea of user preference in mind, the user should also be to adjust the size of the text
on screen to suit their individual needs, just as users are able to choose a language in which to navigate the site.
The group would also like to see a tutorial service provided for the users. Both novice and advanced users of the site
may not know or be aware of the functions which make their library search easier, and by providing a tutorial,

perhaps either by video or PDF, the users would at least have the option to receive help navigating and
understanding the site. This function would be particularly useful to first-year students, English Language Learners,
non-traditional students, and international students, all of which may be new to searching on WorldCat and using
academic language to refine their results. Overall, the group believes that with the preferences being adaptable by
the user and providing explanations both throughout the site and as a function throughout it, the idea of user
friendliness towards the interface will become more positive.

xi

References
Usability Test Participants:

User 1, UNC Charlotte, 2/10/2016

User 2, UNC Charlotte, 2/15/2016

User 3, UNC Charlotte, 2/06/2016

User 4, UNC Charlotte, 2/06/2016

User 5, UNC Charlotte, 2/12/2016

Supporting Documents

Greg Wickliff, Rachael Winterling, Questions from ENGL 4181/5181 Students: Responses from Rachael
Winterling, 2/01/2016

xii