Three common data approaches:
CHAPTER 5: DATA GATHERING, DATA ANALYSIS INTERPRETATION, AND PRESENTATION
1.1 Introduction
Different techniques are available for gathering data. Interviews, questionnaires, and observation are three of
these techniques. Interviews involve an interviewer asking one or more interviewees questions. Questionnaires
are a series of questions designed to be answered asynchronously. Observation involves spending time with
individuals and observing their activities as they happen. All three techniques can be used to collect qualitative or
quantitative data
Data is everywhere. It can be found in numbers, words, measurements, descriptions, comments, photos,
sketches, films, and videos. Data can also be used to understand a design, user needs, and user behavior.
1.2 Five Key Issues
1. Setting Goals
- Gathering data is important. We can learn more about users, their behavior, or their reactions to
technology. There are many different reasons for gathering data, and before starting, we need to
specify which goals we want to achieve.
A good place to start thinking about how to design usable interactive products is to compare
examples of well-designed and poorly designed ones. Through identifying the specific weaknesses
and strengths of different interactive products, we can begin to understand what it means for
something to be usable or not.
2. Identifying Participants
- The goals of the data gathering session will help determine the type of people from which data will be
gathered. This group of people is called the study population. Sometimes it is easy to identify who will
be included in the data gathering-perhaps because there is a small group of users and access to each
one is easy.
3. Relationships with Participants
- One way to ensure that data collected during a study is accurate and relevant is to make sure the
people collecting the data and the people providing the data are familiar with each other.
4. Triangulation
- Triangulation is a process of investigating something from two different perspectives in order to make
sure that the information is accurate. Sometimes, different researchers will collect data and interpret
it in different ways, which is called investigator triangulation. This is similar to using different theories
to understand the data.
5. Pilot Studies
- A pilot study is a small test run of the main study. The aim is to make sure that the proposed method
is feasible before embarking on the real study. This can help identify potential problems in advance,
so they can be corrected. If it is difficult to find participants or access to them is limited, asking
colleagues or peers to participate can work as an alternative for a pilot study.
1.3 Data Recording
Data is collected so that the results of a study can be analyzed and shared. There are different ways to record
data, depending on the goal of the study.
 Notes Plus Photographs
Handwritten notes are a less technical way of recording data that is still very effective. They can be
easily transcribed, and this can help analysts to get a better understanding of the data. Tools are
available to help with data collection and analysis, but handwritten notes are still very effective
 Audio Plus Photographs
Audio recordings are a useful way to record observations and interviews, which is less intrusive than
video and allows observers to focus on the activity. They can be used to gather data in a variety of
ways, and sometimes only a small amount of information needs to be recorded, depending on the
study's goals.
 Video
They are easy to use and less intrusive than setting up sophisticated cameras, but there are occasions
when a video is needed for a long period of time or when holding a phone is unreliable, for example,
when recording how designers collaborate together in a workshop or how teens interact in a
"makerspace." For these kinds of sessions, more professional video equipment that clearly captures
both visual and audio data is more appropriate.
1.4 Qualitative Analysis
Interviews
- An interview is a conversation where the interviewer has some control over the direction. Interviews
can be open-ended (informal), structured (formal), or semi-structured (in the middle). Group
interviews are the most structured.
Four Main Types of Interviews:
Open Ended / Unstructured
Unstructured interviews are a qualitative research method that can be natural and spontaneous but risky due to
difficulty in comparing data and potential interviewer bias. They are often used in exploratory research for
gaining a deeper understanding of a phenomenon or issue.
Structured Interviews
Structured interviews are predetermined questionnaires used with each participant in a study, useful for specific
topics or assessing a population. For instance, questions may inquire about frequently visited websites, online
purchase habits, or average purchase frequency.
Semi-structured Interviews
Semi-structured interviews combine structured and unstructured questions, covering same topics with a basic
guide. The interviewer asks preplanned questions and probes the interviewee for more information until no new
relevant information is provided.
1.5 Questionnaires
Questionnaires are surveys that collect data from a large number of people using closed or open-ended
questions. They are efficient because they can be distributed widely and don't require additional resources.
Well-designed questionnaires can efficiently gather specific information, and they can be used alone or with
other methods.

Questionnaire Structure
Questionnaires gather information on a person's experience and perspective. They are useful for understanding
context and can be distributed to a large number of people. When designing a questionnaire, consider question
order, length, and relevance. Layout and pacing should also be considered.
Question and Response Format
Question and answer formats can differ significantly and affect the clarity of responses. For example, some
formats allow for a single answer, while others permit multiple responses. Ranging questions can be useful in
specific situations. Using the appropriate format can help to obtain clear answers from respondents.
Check Boxes and Ranges
Be careful when providing predefined response options to avoid confusion, such as by ensuring that there is no
overlap between response options. It is also important to consider the specific purpose of the question and to
tailor the response options accordingly.
Rating Scales
Different rating scales serve different purposes. Likert scales ask respondents to identify a set of statements
representing a range of possible opinions, while semantic differential scales rely on pairs of words representing
the range of possible opinions. Likert scales are more commonly used because finding consistent statements is
easier than finding semantic pairs.
Likert Scales
A Likert scale is a survey tool that measures attitudes and opinions on a topic. Respondents are asked to indicate
their level of agreement or disagreement with a series of statements, which are then assigned numerical values
ranging from 1 to 5.
Semantic Different Scales
The Semantic Differential Scale measures attitudes and perceptions toward a concept by presenting polar
adjectives and asking participants to rate the concept on a scale between them.
Administering Questionnaires
Questionnaires require a high return rate to be effective, with a 40 percent return rate being acceptable for
smaller sample sizes, while larger or more remote populations may have lower return rates.
Online questionnaires have advantages over paper questionnaires, such as faster data collection, skip logic, and
automated analysis.
Online questionnaires may have limited generalizability due to difficulties in obtaining a random sample, and
smartphone surveys may not be applicable in all areas. To create an online survey, plan the timeline, design the
questionnaire, program it, and recruit respondents.
1.6 Observation
Observation is a useful tool throughout product development. Early in design, it helps designers understand
users' context, tasks, and goals. Later in development, for example, in evaluation, observation may be used to
investigate how well a prototype supports these tasks and goals.
Direct Observation in the Field
Observation in the field can be helpful in understanding how users interact with technology, but it can be
difficult to do well. There is a balance to be struck between following the goals of the observation and adapting it
as new information is learned.
Structuring Frameworks for Observation in the Field
Observation can be structured in a simple way by focusing on three key points: what you see, what you hear,
and what you feel.
Who is using the technology at any given time? Where are they using it? What are they doing with it?
Degree of participation
The degree of participant involvement in studies varies depending on the study's goals and practical and ethical
considerations. Passive observation is the most passive form of participation, but it's difficult to implement in the
field.
Ethnography
Ethnography is a method used in the social sciences to study the organization and activities of societies. It has
gained credibility in the design of collaborative systems and is used in particular to study the perspectives of
participants in a situation.
Direct Observation in controlled environments
Controlled observation is formal and scripting can create more credible results by ensuring that all participants
are treated the same. The setup of equipment and personnel is crucial to get a complete picture of the
participant's activity, such as recording facial expressions and body language with multiple cameras.
The Think-Aloud Technique
a cognitive psychology and user experience research method where participants verbalize their thoughts and
feelings while performing a task to gain insight into their decision-making process.
Indirect Observation: Tracking Users’ Activities
Sometimes it's not possible for observers to be present during a study, so data is collected by tracking what
people do indirectly. This could involve keeping track of diaries or logs of what people say and do.
Diaries
Diaries track experiences, thoughts, and feelings and are used to gain insights into the experiences and
behaviors of individuals over time, in social science research
1.7 Choosing and Combining Techniques
There are many different ways to collect data, and the best way to collect data depends on the study's goals and
the participants' characteristics. Some of the most common data-gathering techniques includes interviews, focus
groups, and questionnaires.
In addition, technique choice is influenced by practical issues.
The focus of the study.
 What kind of data will support the focus and goal of the study? This will be influenced by the
interaction design activity and the level of maturity of the design.
 The participants involved. Characteristics of the target user group including their location and
availability.
 The nature of the technique. Does the technique require specialist equipment or training. and do the
investigators have the appropriate knowledge and experience?
 Available resources. Expertise, tool support, time, and money.
Part 2 Data Gathering, Data Analysis Interpretation, and Presentation
2.1 Introduction
Data analysis begins with initial reactions or observations from the data, followed by more detailed work using
structured frameworks or theories to support the investigation. It is important to look over the data to check for
any anomalies, and it is important to make sure that the data supports any hypotheses. A common mistake is
allowing existing beliefs or biases to influence the interpretation of results.
2.2 Quantitative and Qualitative
Quantitative data
is in the form of numbers or data that can easily be translated into numbers. Examples are the number of years
of experience the interviewees have, the number of projects a department handles at a time, or the number of
minutes it takes to perform a task.
Quantitative analysis uses numerical methods to ascertain the magnitude, amount, or size of something, for
example, the attributes, behavior, or strength of opinion of the participants.
Qualitative data
is in the form of words and images, and it includes descriptions, quotes from interviewees, vignettes of activity,
and photos. It is possible to express qualitative data in numerical form, but it is not always meant to do so.
The qualitative analysis focuses on the nature of something and can be represented by themes, patterns, and
stories. For example, in describing the same population, a qualitative analysis might conclude that the average
person is tall, thin, and middle-aged.
Initial Analysis Steps for Data Typically Collected through: interviews, questionnaires, and observation
2.3 Basic Quantitative Analysis
Cairns (2019) provides two basic quantitative analysis techniques for interaction design: averages and
percentages.
Percentages are used to standardize data, such as comparing two or more large sets of responses.
Averages and percentages are numerical measures.
Three different types of averages:
 Mean is the commonly understood interpretation of average, meaning by adding together all figures
and dividing by the number of figures with which they started.
 Median and mode averages are less well-known but are very useful. The median is the middle value
of the data when the numbers are ranked.
 Mode is the most commonly occurring number. For example, in a set of data, the median is 6 and the
mode is 7, while the mean is 50/9 = 5.56.
2.4 Basic Qualitative Analysis
The first step in qualitative analysis is to gain an overall impression of the data and to start looking for interesting
features, topics, repeated observations, or things that stand out. Some of these will have emerged during data
gathering, and this may already have suggested the kinds of patterns to look for, but it is important to confirm
and re-confirm findings to make sure that initial impressions don't bias analysis. Qualitative analysis involves
looking at data in a different way than quantitative analysis does.
Identifying Themes
Thematic analysis is a technique used to identify patterns or themes in qualitative data. It is important to check
for consistency and find further themes that may not have been noticed the first time. Affinity diagrams are a
way to organize ideas and insights into a hierarchy and can be used to explore data and look for an overall
narrative. The study's goal is to provide an orienting focus for the identification and formulation of themes in the
first and subsequent passes through the data.
Categorizing Data
When studying something, like how people use navigation aids in an online educational setting, Inductive
analysis is appropriate because it helps to uncover the data's themes by looking at what the people are doing.
Sometimes, the analysis frame is chosen beforehand, based on the study goal. In that case, analysis proceeds
deductively. For example, in the study by Nicole Lotz et al. (2014), which looked at novice interaction designers
in Botswana, the researchers used a predetermined set of categories based on Schön's design and reflection
cycle (naming, framing, moving, and reflecting). This allowed the researchers to identify detailed patterns in the
designers' behavior, which provided implications for education and support.
Critical Incident Analysis
Critical incident analysis helps to identify significant data subsets for detailed analysis by reporting facts about
behavior and limiting reporting to behaviors that make a significant contribution. It is used in interaction design
to identify specific incidents and analyze them in detail, with other data used as context. Users or observers may
identify incidents, such as when users are stuck in an evaluation study.
2.5 What kind of Analytic Framework to Use?
Conversation Analysis
Conversation analysis is a technique used in sociological studies to analyze how conversations start and how
turn-taking is structured. It has been used to analyze interactions in a range of settings, and to compare
conversations that take place through different media. Voice assistants, such as Amazon Echo, have become
increasingly popular in domestic settings, but how do families orient and adapt to them? Does using this device
change the way they talk, or do they talk to the device as if it was another human being? Martin Porcheron et al.
(2018) conducted a study examining how such devices were being used by families in their own homes, and they
used conversation analysis with excerpts from selected conversations.
They used a particular type of syntax for marking up the minutiae of interactions and speech that took place
during an approximate 10-second period. This level of detail enables the analysis to reveal subtle cues and
mechanisms that are used during the conversations.
Discourse Analysis
Discourse analysis is a way of understanding what people are saying in a specific situation. It can be used with
data from interviews, social media, and emails. For example, Carlos Roberto Teixeira et al. used discourse
analysis to understand the conversations happening online during the political scandals in Brazil in 2016. They
scraped the raw data of the tweets, ranked them, and then selected 100 of the most influential. They then
manually classified the tweets according to a number of themes, including "support," "criticism and protest,"
"humor," and so on.
Content Analysis
Content analysis is a technique that helps us to understand what kinds of things people are talking about on the
internet. We can use it to analyze the text of tweets, links, animated gifs, videos, and images. For example, Mark
Blythe and Paul Cairns (2009) analyzed 100 videos from a YouTube search by relevance for "iPhone 3G ❞. They
classified the videos into seven categories: review, reportage, "unboxing," demonstration, satire, advertisement,
and vlog commentaries (such as, complaints about queues).

Interaction Analysis
Interaction analysis is a way of studying how people interact with each other and their environment. It is based
on video recordings, and it uses both talk and nonverbal interactions with artifacts and technologies. The goal is
to learn generalizations about how people make sense of each other's actions and how they achieve collective
goals.

Grounded Theory
Grounded theory is a method of studying data that is based on systematic analysis and interpretation of what
has been observed. It is an inductive approach to developing theory, which starts by analyzing data to find
themes and then using that analysis to collect more data to further develop the themes. The goal is to develop a
theory that is based on the data, and the data collection process is driven by the emerging theory.

Systems-Based Frameworks
When researchers want to understand how a new technology should be introduced and how it will affect
society, they need to look at data from a long period of time. This data can be analyzed in a variety of ways, but
understanding how a whole socio-technical system works at scale requires a different kind of analysis.

2.6 Tools to Support Data Analysis

Tools that can be used to support the organization of data (for example, by providing facilities for cat-
categorization, theme-based analysis, and quantitative analysis) are often very effective at helping to speed up
data analysis, while more sophisticated tools are available to support the coding and manipulation of data, as
well as the performance of statistical tests.

Nvivo is a software tool that can be used for data annotation, coding, and statistical analysis. Dedoose is a similar
software tool, but it is designed specifically for the analysis of big data. SAS and SPSS are popular quantitative
analysis packages that can be used for statistical analysis.

2.7 Interpreting and Presenting the Findings

There are a variety of ways to present data, depending on the analytical approach and audience. This can include
using tables of numbers, diagrams, themes, or summaries. It's important to choose an appropriate style based
on the data gathering and analysis techniques used, as well as the original goals of the study.

Structured Notations
There are many notations that can be used to describe interaction design. Some, like the job models emphasized
in contextual design, are simple and easy to understand. Others, like the modeling language Unified Modeling
Language (UML), are more exact and demand a deeper level of comprehension. , using a structured notation in
combination with other formats, such as stories, can help overcome these limitations.

Using Stories
Stories are a popular way to communicate ideas and experiences and can be used in interaction design to gather
further data. Ethnographic field studies, in particular, are a good source of stories.

Summarizing the Findings

The presentation of research findings can help to ensure that the findings are understood correctly, and can
include graphical representations, data excerpts, and more. It is important to use appropriate analysis
techniques to avoid over-emphasizing the findings or misrepresenting the evidence.