Usability Research Plan

Version 2

Allen Scott

Feb. 6, 2023
Table of Contents

Usability Research Plan.........................................................................................................................................................

Table of Contents........................................................................................................................................................
Summary.........................................................................................................................................................
Research Issues and Questions.......................................................................................................................
Research Structure..........................................................................................................................................
Schedule..........................................................................................................................................................
Resources........................................................................................................................................................
Deliverables....................................................................................................................................................
References......................................................................................................................................................

Summary

The University of North Florida Thomas G. Carpenter Library’s website (“the product”)

attempts to meet the needs of a wide academic audience. A usability test is needed to understand

how it is functional, efficient, and desirable. The test will be conducted by Allen Scott (“the

tester”) and centered on professors (“the users”) from the University of North Florida (“the

organization”), who have unique experiences with the site as well as feedback from their classes

with university students. Expected results are a well-rounded state of usability already in an

iterative process of design for stakeholders; otherwise, the recommendations of this research will

attend to the findings.

This research plan outlines an effort to discover the needs and goals of university

professors regarding the UNF Library (https://www.unf.edu/library/) through rapid user research
on the library’s web-based product, and it presents a schedule designed to meet this goal. It

includes plans for internal discovery to identify UNF’s requirements and constraints; usability

testing and diary studies of the existing product to uncover interaction problems, current

experiences, and current solutions; four to five rounds of survey to determine demographic

segmentation and technology use of the existing and potential user base; and (optionally) two to

three rounds of focus groups to determine whether students feel the proposed features of

solutions by professors will actually help them. Data analysis will follow in any case. A schedule

of all research through the week of March 12th is included, and in place of a budget, a list of

used resources is proposed.

This research plan is valid between 2/06/2023 and 3/20/2023, at which point an updated

Usability Testing Plan will be submitted.

Research Issues and Questions

For its users, is the site functional (i.e., able to do a set of things), efficient (ie, loads

quick and easy to navigate), and desirable (i.e., has a positive “look and feel”)? The site is

functional if the professors consider it useful. The site’s complexity, or the incomprehensibility

of its interface, may conceal its key features. The site is efficient if it works well. Professors

value their time, so speed and ease of operation are important. Finally, the site is desirable if it

allows pleasure and happiness through its use. The website must make sense in terms of its

cultural web of meaning.

 How can iterative development work by continual refinement and through trial and error?

Is UNF’s model “corporate” (someone/group decides what must be done and others build it)? Is

it “waterfall” (an extensive requirement document that specifies every detail of the final product,

collecting the document author’s gut-level guesses and closed-door debate)? While “corporate”

can be considered an inappropriate term for a public institution, the model may accurately

describe how a team or panel determines all the decisions of the site.

How can the stakeholders be identified? Which departments have professors who own

more of a stake in the library website than others? What do they consider to be the most

important issues; who is left out of decisions? As a secondary line of inquiry, these questions

examine the politics behind the library. Students benefit from the library as do the faculty, and it

is possible that others, like non-academics, receive less benefit.

Here are key questions to ask stakeholders: In terms of what you do on a day-to-day

basis, what is the goal of UNF’s library website? Are there ways that the library is not meeting

those goals? If so, what are they? Are professors teaching undergraduate or graduate? What extra

roles? Instructor, Assistant Professor, Associate Professor, or Professor? What role do

demographics or departments/fields play? Different approaches to students, e.g., freshmen versus

upperclassmen? What are your/their usage habits? What reoccurring issues and student feedback

do professors encounter? If new, can they participate in observational research? What other user

experience knowledge is floating around UNF, answering research questions without requiring

any original research? What surveys done by marketing, customer support feedback summaries,

interviews the by development group, and unused reports from usability consultants? Where can

this information be collected and integrated? These questions will allow entry into the iterative

development of the library website so usability research can begin.

Research Structure

The following structure exemplifies the research process and stops at the creation phase

in recognition of this project’s limits. First, there is an examination phase. We define the

problems and whom they affect. Questions are raised, needs are analyzed, information is

collected, research is conducted, and potential solutions are evaluated. Strengths and weaknesses

are enumerated and prioritized. Users’ needs and their capabilities are studied, and the website is

evaluated. For this goal, we speak to the stakeholders, namely the UNF Library.

Next, in a definition phase, solutions are specified. Changes in the website are mapped

out with even greater detail as additional information about the real needs and capabilities of the

target audience is uncovered. Ideas can be modified when options are proposed to the UNF

Library, which may include rewriting the product description or detailing the problems usability

testing solves (perhaps listing problems outside its service or scope).

Finally, there may be a creation phase, but this phase is beyond the limits of this research.

The UNF Library may specify solution plans in a continuation of the project, using the new

product definition, rewriting the product description, or reflecting its new purpose and new

knowledge of the user’s needs. It may outline the new questions that need to be answered,

markets that need to be investigated, and areas that need to be focused on in the next round of

research.
Schedule

A tentative schedule will be followed by scheduling at least one and at maximum two

interviews with professors per week up to March 20th. The proposed date for the launch of an

email to professors with time slots to be filled is February 7th, wherein surveys will be

conducted. The proposed date for first objective notes on internal discovery, usability testing,

and diary studies is February 10th. The proposed date for each modified survey is one day before

each interview.

All surveys and research must be completed before these due dates: Feb. 20 is the

completion date of Personas (i.e., realistic representations of fictional potential users) and

Scenarios (e.g., of imagined use). By this time, diary studies have begun a record of long-term

studies of users, investigating processes and activities over time, and how users’ views and use

patterns change with experience. Surveys have begun to quantitatively describe the audience,

segment them into subpopulations, and investigate their perceptions and priorities. The user

market has been defined by turning user research into fictional characters in order to understand

the needs of different users. Other important dates for this project are Mar. 20 (User Testing

Plan), Apr. 10 (User Testing Report), Apr. 17 (Presentations), and May 4 (Portfolio and Short Lit

Review).

Resources

The “budget” (i.e., a list of used resources) is proposed in terms of time rather than

money: preparation for a single research project, 10 hours; recruiting and scheduling, 2-3 hours
per person recruited; usability test or report for online delivery, 12 hours; conducting research, 5

hours per person (or as available by professors); analyzing results, 5 hours per person; and

preparing the one hour presentation, 6 hours.

A prioritization exercise can assist in tackling research. It will analyze the UNF Library

site usability objectively and local competitors’ sites: Jacksonville University, Edward Waters

University, Trinity Baptist College, and Flagler College. It will find who is in charge of the

product (refer to Goodman 73). It will peruse the site's current documentation and talk with tech

support (refer to Goodman 243). It will find prior research online (refer to Goodman 211). It will

survey professors on the site’s usability (refer to Goodman 95). Lastly, it will analyze this

qualitative data (refer to Goodman 423).

Usability Testing will prepare a questionnaire for interviews (refer to Goodman 183 and

327) and focus on how well people understand the elements of the interface, their expectations

for structure and functionality, and how they perform key tasks. The software for usability

testing interviews will assist in conducting 4-5 one-on-one structured, task-oriented interviews

with users.

Two final brainstorming exercises are serving as support. A “Rewriting Goals as

Questions” exercise starts by rewriting the goals raised during interviews as simple, user-specific

questions or information to be gathered. It then broadens narrow questions into general topics to

get at the root causes of the problems (e.g., “Why does that feature exist?”). A “Expand General

Questions with Specific Ones” exercise fleshes the larger, abstract questions into specific ones

that can be answered by research (in the next interview).

Deliverables

Success will be measured in identifying what is problematic and what is not. Several

things probably will come up again and again, like a certain function, critical idea, lack of

interest, or high interest. These discoveries will reveal “the ease with which the product is

integrated into the corporate brand” (48 Goodman). They will enable recommendations to be

made.

We are delivering a list of ways or issues in which the product’s user experience impacts

the organization. Each issue represents a goal for the research program; it focuses the research

plan and helps uncover how the product can be improved for the greatest benefit to the

organization. Adding communal goals, such as supporting environmental sustainability, doing

political advocacy, or promoting public health, can lead to a “double bottom line” and hence

another tension. This and other tugs of war are kept in mind to consider how the organization has

resolved them in the past.

Potentially, the UNF Library can plan another iterative cycle after findings are offered to

UNF’s product development, product marketing, or customer support or policy team. However,

the main deliverables are projects based on personas, a test plan, and a test report are uploaded to

an online portfolio. The personas research opens the project to acknowledging that the actual

users are different from those imagined and not all the same. The personas represent goals and

behavior patterns rather than demographic attributes or the job responsibilities of the professors.

Patterns through surveys and usage data can consider what aspects of users tend to drive their

goals and needs. This task will drive the selection of the UNF Library interface’s evaluators, who

will resemble the product’s imagined audience. Choosing the most representative group can
reduce the amount of research done and focus the results, and one or two people from secondary

target audiences can test for a radically different perspective in those groups.

Testing the present version of the UNF Library website will constitute an investment in

its next version. Tasks are expected to be representative of typical user activities, and participants

may describe a recent situation they found themselves in that the product could address. The test

report will comprise the most natural and complete responses of evaluators.

References

Goodman, Elizabeth, et al. Observing the User Experience: A Practitioner’s Guide to User

Research. Elsevier Science, 2021.