SIA 2101

RESEARCH METHODS

Lecture 10: Research Analysis & Interpretation

Dr. Phil Nyoni

 Session Objectives

• Prepare and organize the data for analysis

• Types of quantitative data used in assessment
• Explore and code the data
• Develop description and themes from the data
• Represent and Visualise Data
• Interpret and Report the findings
• Validate the accuracy and credibility of the
findings
 Data analysis
• The way information and results are interpreted
and assessed
– Assigning meaning to figures, stories, observations,
etc that have been gathered and recorded.
– Conceptual frameworks guide data analysis.
– Data analysis possible by hand or computer (various
packages, e.g. SPSS; etc.)

Analysis is the search for patterns in data and for ideas

that help explain why those patterns are there in the
first place.
 Data analysis
• Data analysis helps discover and substantiate
patterns and relationships, test our expectations,
and draw inferences that make our research
fruitful.
• Answer research questions or test hypotheses.
• Theory and results of prior research should
guide our statistical choices, as they have
guided the choice of other research methods.
 Data analysis
• Statistics play a key role in achieving valid research
results in terms of measurement, causal validity and
generalisability.
• Some statistics are useful primarily to describe the results
of measuring single variables. E.g. frequency
distributions, measures of central tendency and
dispersion.
• Other statistics are useful primarily in achieving causal
validity. E.g. cross tabulations, regression.
• It is possible also to estimate the degree of confidence
that can be placed on generalisations from a sample to
the population.
 Possible Biases
Problems with data collection
• Researcher biases
• Respondent biases
• Instrument biases
Data analysis biases
• Coding errors
• Data punching & input errors
• Inappropriate statistical analysis
Biases (subjectivity) in intepretation of results
 Quality control measures
• Thorough training of staff plus pre-testing of tools
(interpretation of the questionnaires, if necessary)
• Standardization tests- Intra-personal/ interpersonal
errors
• Close monitoring of the field work by qualified persons
• Cross-checking of the field questionnaires for anomaly
daily
• Daily review of enumerator experiences and problems
• Progress review per plan and by checklist
• Data cleaning: collection, entry,
• Integrity of equipments: maintain accuracy using known
weights
 Handling data before analysis
• Clearly identify source (by name or code)
• Keep track of those who have not responded and
follow up
• Indicate the date and file data securely
• Review responses for completeness
• Translate into code (if necessary) or summarise using
key words
• Decide on how to record missing data
• Transfer data to blank copies of the original monitoring
sheet or a spreadsheet programme in preparation for
analysis.
 Steps in data analysis and interpretation

1. Review the questions that generated the

information.
• Why was the particular information necessary? What
kind of decisions are to be made based on this
information?
2. Collate the relevant data:
– Baseline info and previous surveys or assessments
undertaken
– Background info e.g. morbidity data, food security info,
health facilities data, ongoing interventions, security
situation.
– Sort information into parts that belong together.
Steps in data analysis and interpretation

3. Data preparation and cleaning

– Before starting the analysis, the data needs to be
prepared and “cleaned”. Issues to look out for include:-
– Missing data
– Data out of the required range.
– Extreme (biologically unlikely) weight for height data –
outliers
4. Analyze quantitative data
5. Analyze qualitative data
6. Integrate the information
 Basic Types of Data
Categorical
•Nominal
– (no inherent order)
– city names, types of diseases, ...
•Ordinal
– (ordered, but not at measurable intervals)
– first, second, third, …
– cold, warm, hot
•Continuous
– Ratio: consistent distance between each point
– Interval: there is a zero starting point i.e. list of integers or reals
 Analysing quantitative data

• First thing to do to analyse quantitative data is

convert raw data into useful summaries
– Descriptive measures
• Proportions, frequencies and ratios
– Measures of central tendency
• Mean/average, median, mode
– Measures of dispersion
• Range, standard deviation, percentiles.
 Descriptive statistics

Descriptive Statistics

Central Tendency Variability Relative Standing

Mean Variance Z-Score

Median Standard Deviation Percentile Ranks
Mode Range
 Conduct hypothesis tests

A hypothesis is less general than a theory.

• Theories and hypotheses describe causal
relationships between two or more variables.

A theory is a fairly general explanation of some

political phenomenon.
• Theories must be testable
 Testing Hypotheses
Using appropriate statistical analysis, test
hypotheses, eg:
• t-test to test the significance of differences of the means
of two groups
• Analysis of variance (ANOVA) to test significance of
differences among the means of more than two different
groups, using the F test
• Using regression analysis to establish the variance
explained in the DV through independent variables
• Compute the sample statistic
• Make a decision about rejecting/failing to reject
 Inferential statistics

Inferential Statistics

Continuous (iv) Continuous (iv)- Continuous (iv)- Continuous (iv)-

Continuous (dv) Continuous (dv) Continuous (dv) Continuous (dv)
Chi-Square
Analysis
Pearson Mann- Discriminant Point
T-Test Phi
Correlation Spearman Witney Analysis Biserian
Coefficient Rho U-Test Correlation Coefficient
Analysis
of
Regression Kendall’s Variance Kruskall
Coefficient Tau Wallis
Test
Analysis
of
Friedman
Covariance
Two-Way
Anova
How to select an appropriate statistic

• Determine the type of quantitative research

question or hypothesis you want to analyze
• Identify the number of independent variables
• Identify the number of dependent variables
How to select an appropriate statistic

• Consider the scale of measurement for your

independent variable(s) in the research
question or hypothesis
• Identify the scale of measurement for the
dependent variables (e.g. continuous or
categorical)
• Determine if the distribution of the scores is
normal or skewed
 Univariate analysis
• Simplest form of quantitative analysis – involves
describing a case in terms of a single variable.
E.g. gender – we would look at how many
women and men there are.
• Graphs and frequency distributions are the two
most popular approaches of demonstrating
univariate analysis.
 Univariate analysis
• Three features of the shape are important in a
univariate analysis:
a) Central tendency – the most common value or
the value around which cases tend to center.
b) Variability—the extent to which cases are
spread out through the distribution or clustered
in just one location.
c) Skewness—the extent to which cases are
clustered more at one or the other end of the
distribution of a quantitative variable.
 Selecting a statistical program
• Statistical Package for Social Sciences (SPSS)
most popular
• Other programs
– Mini-tab
– Statview
– SAS (JMP/JMPIN)
– StatPac
• Use PC or Mac platforms
 Excel
• Spreadsheet is a computerised ledger
• Divided into rows and columns
– Columns identified with alphabetic headings
– Rows identified with numeric headings
• Common user interface on all Office applications
– Menus and toolbars are similar to Word and Power Point
• Worksheet is an Excel spreadsheet
• Workbook contains one or more worksheets
• Cell references
• Constants--entries that do not change
• Formulas--combination of constants and functions
Software: Microsoft Excel
 SPSS
SPSS is Statistical Package for the Social Science
• It is used by market researchers, health researchers,
survey companies, government, education
researchers, marketing organizations and others
Statistical packages, e.g. SPSS
• You can access, manage, and analyze enormous
amounts of data with SPSS.
• SPSS offers statistical test for frequency distributions,
rank correlations, regression analysis, and cluster
analysis
SPSS
 Quantitative data analysis
Making sense of the numbers to permit meaningful
interpretation

It involves:
1. organising the data
2. doing the calculations
3. interpreting the information
• lessons learned
4. explaining limitations
 Organising the data

• Organise all forms/questionnaires in one

place
• Check for completeness and accuracy
• Remove those that are incomplete or do not
make sense; keep a record of your decisions
• Assign a unique identifier to each
form/questionnaire
 Enter your data

• By hand
• By computer
– Excel (spreadsheet)
– Microsoft Access (database mngt)
– Quantitative analysis: SPSS (statistical
software)
 Do the calculations

• Count (frequencies)
• Percentage
• Mean
• Mode
• Median
• Range
• Standard deviation
• Variance
• Ranking
• Cross tabulation
 Which calculation do I use? It depends
upon what you want to know.

Do you want to know how many individuals Frequency

checked each answer?
Do you want the proportion of people who Percentage
answered in a certain way?
Do you want the average number or average Mean
score?
Do you want the middle value in a range of values Median
or scores?
Do you want to show the range in answers or Range
scores?
Do you want to compare one group to another? Cross tab
Do you want to report changes from pre to post? Change score
Do you want to show the degree to which a Standard deviation
response varies from the mean?
 Interpreting the information

Numbers do not speak for themselves.

For example, what does it mean that 55 youth
reported a change in behavior. Or, 25% of
participants rated the program a 5 and 75%
rated it a 4. What do these numbers mean?
Interpretation is the process of attaching
meaning to the data.
 Discuss limitations

Written reports:
• Be explicit about your limitations
Oral reports:
• Be prepared to discuss limitations
• Be honest about limitations
• Know the claims you cannot make
– Do not claim causation without a true
experimental design
– Do not generalize to the population without
random sample and quality administration (e.g.,
<60% response rate on a survey)
 Qualitative data analysis

1. Any type of research that produces findings

not arrived at by statistical procedures

2. Persons, lives, experiences, emotions

3. Qualitative data analysis – an inductive

process of organising, classifying or coding
data into categories / themes and identifying
patterns among them
 Qualitative Data Analysis Strategies

• Identifying themes
– Begin with big picture and list “themes” that emerge.
• Events that keep repeating themselves
• Coding qualitative data
– Reduce data to a manageable form
– Often done by writing notes on note cards and sorting
into themes.
• Predetermined categories vs. emerging categories
 Coding
• Coding is a process of reducing the data into
smaller groupings so they are more manageable.
• The process also helps you to begin to see
relationships between categories and patterns of
interaction.
 Coding
• Sections of text transcripts may be marked by the
researcher in various ways (underlining in a
colored pen, given a numerical reference, or
bracketed).
Selecting a text analysis program
Software: ATLAS.ti

http://www.atlasti.com/ -- free trial available

 Categories/Themes
• A major step in analyzing qualitative data is
coding speech/ words/text into meaningful
categories/themes.
• As you read and reread through the data, you
can compile the data into categories or themes
 Categories/Themes
• A theme/category is generated when similar
issues and ideas were expressed by participants.

• The theme or category may be labeled by a word

or expression taken directly from the data or by
one created by the researcher because it seems to
best characterize the essence of what is being
said.
 Steps in Coding the Data
• Get a sense of the whole
• Pick one document (e.g. one interview, one field
note….). Go through it, asking the question “what is
this person talking about?”
• Identifying text segments, placing a bracket around
them and assigning a code word or phrase that
describes the meaning of the text
 Steps in Coding the Data
• After coding an entire text, make a list of all the
code words. Group similar codes and look for
redundancy codes
• Take the list and go back to the data. Circle
specific quotes from participants that support the
codes
• Reduce the list of codes to get five to seven
themes/categories
 Making coding manageable
• Make photocopies of original data
– Why?
• Read through all of the data.
– Attach working labels to blocks of text
• Cut and paste blocks of text onto index cards.
• Group cards that have similar labels together
• Revisit piles of cards to see if clusters still hold
together.
 Other Strategies
• Concept Mapping
– Analyzing Antecedents and Consequences
– Displaying Findings
– Stating what’s missing

Illness

Social Skills
Absenteeism
School Safety
Types of grounded theory designs: The
systematic design

• Open Coding: properties and dimensionalised

properties
• Axial Coding: researcher selects one open
coding category and places it at the center as
the Central Phenomenon and then relates all
other categories to it.
• Selective Coding: writing a theory based on
the interrelationship of the categories from
axial coding
What is the Process of Data Analysis?

Codes the Text for Codes the Text for

Description to be Used Themes to be Used
in the Research Report in the Research Report

The Researcher Codes the Data (i.e., locates text

segments and assigns a code to label them)
Interactive Simultaneous
The Researcher Reads Through Data
( i.e., obtains general sense of material)

The Researcher Prepares Data for analysis

( e.g., transcribes fieldnotes)

The Researcher Collects Data (i.e., a text file, such as

fieldnotes, transcriptions, optically scanned material)
 How do You Prepare and Organize the
Data?
• Develop a matrix or table of sources that can be
used to organize the material
• Organize material by type
• Keep duplicate copies of materials
• Transcribe data
• Prepare data for hand or computer analysis (and
select computer program)
How do you Explore and Code the Data?

• Obtain a general sense of the data by

performing a preliminary exploratory analysis
• Memo ideas
• Consider whether more data are needed
• Coding the data
A Visual Model of the Coding Process in
Qualitative Research
Divide text Label Reduce Collapse
Initially read
into segments segments of Overlap and codes into
through data
of information information redundancy themes
with codes of codes

Many Many Codes

Pages Segments 30-40 reduced
codes Reduce Codes to
of Text of Text to 20 5-7 Themes
 How do You Use Codes to Build
Description?
• Describe
– People
– Events
– Activities
– Processes
• Describe in detail
 How do You Use Codes to Identify Themes?

• Ordinary themes
• Unexpected themes
• Social science themes
• Layering and connecting themes
 Coding Used in Theme Passage

Safety
Title for theme The violence in the city that involved university students and the subsequent
based on words of
gun incident that occurred in a campus classroom shocked the typically tranquil
participant
campus. A counselor aptly summed up the feelings of many: “When the
students walked out of that classroom, their world had become very chaotic; it
had become very random, something had happened that robbed them of their
Evidence for themes sense of safety.” Concern for safety became a central reaction for many
based on multiple informants.
perspectives of When the chief student affairs officer described the administration’s reaction to
participants the incident, he listed the safety of students in the classroom as his primary
goal, followed by the needs of the news media for details about the case,
Within themes
helping all students with psychological stress, and providing public information
are sub-themes
on safety. As he talked about the safety issue and the presence of guns on
campus, he mentioned that a policy was under consideration for the storage of
guns used by students for hunting. Within 4 hours after the incident, a press
conference was called during which the press was briefed not only on the details
of the incident, but also on the need to ensure the safety of the campus. Soon
thereafter the university administration initiated an informational campaign on
campus safety. A letter, describing the incident, was sent to the university board
members. (One board member asked, “How could such an incident happen at
this university?”)
 Content analysis
These packages are useful tools for breaking into text. Most have the capacity
to undertake:

• word frequencies indicating how often each word occurs in a document;

• category frequencies where synonyms are grouped into categories and

the program shows how many times each category occurs in the document;

• KWIC (key word in context) which displays, in alphabetical order, each word
together with a number of words on either side to provide information on its
context in the document;

• cluster analysis where groups of words can be identified as being utilised

in similar contexts

• co-occurrence of pairs of words. The more sophisticated of these

programs are developing the capacity to attempt semiotic analysis and often
include cultural grammars.
 Exploring Data

• The first step in data analysis is to

explore the data
• Read through data and make sure all
information is complete and legible
before proceeding to analysis
• Obtain a general sense of the data
• Memoing - read and write memos about
all field notes, observer comments to
get an initial sense of the data
 When does one start with data
analysis in qualitative research?
1. Data analysis is an ongoing process that routinely starts
prior to the first interview
2. When the process of data collection begins, they start with
conducting interviews or observations
3. In practice ~ Data analysis tends to begin when data
saturation becomes noticeable
4. Data Saturation ~ Patterns & themes start to recur, and no
new information is added by more interviews.
5. At the end of data collection, data analysis starts to take all
information into account, taking on a more directed form.
Step-wise plan for qualitative data
analysis

The collected data must be organised and

prepared for analysis
– Transcribing the interview word-for-word,
– Optically scanning material,
– Typing field notes
– Sorting and arranging the data into different types
depending on the sources of information
 Analysing Qualitative Data
• Describe the phenomena
– transcribe all interviews/observations
• thorough and comprehensive (‘thick’ description), i.e., information
about the context of an act, the intentions of the actor and the
process in which action is embedded.
describe the sample population,
– who were the key informants, what made them qualify as
such? Who took part in the FGDs? How representative were
the participants of the groups they represented? Under what
circumstances were observations carried out? Who was
observed (and who was not)?

• Classification of the data

– look for and code key words and phrases that are similar in meaning
– categorize issues by topics
• Identify and group (categorise) pieces of data together, i.e.,
separate similar or related data
 Analysing Qualitative Data continued ...
• Interconnect the concepts
– compare responses from different groups
– determine patterns and trends in the responses from different groups or
individual respondents
– make summary statements of the patterns or trends and responses
– cite key quotations, statements and phrases from respondents to give
added meaning to the text.
– re-check with key informants to verify the responses and the generalization
of the findings.

Display summaries of data in such a way that interpretation becomes

easy,
• list the data that belong together – may be followed by further
summarization graphically in some chart (i.e., a matrix – most common
form of graphic display of qualitative data) or a figure (i.e., diagram, flow
chart). These help visualize possible relationships between certain
variables.
Analysing Qualitative Data continued ...
• draw conclusions, and (remember…)
• collection, processing, analysis and reporting of qualitative data are
closely intertwined, and not (as is the case with quantitative data)
distinct successive steps. One searches for evidence, purposively
looks for associations during the fieldwork by intertwining data
collection and analysis, verifies findings by looking for independent
supporting evidence.
• develop strategies for testing or confirming findings to
prove their validity.
• Check for representativeness of data (since informants are selected
systematically & according to previously established rules) --- are all
categories of informants been interviewed? Cross-check data with
evidence from other, independent sources (informants, informant
categories or different research techniques)
 Data Analysis During Collection
• Analysis not left until the end
• To avoid collecting data that are not important
the researcher must ask:
– How am I going to make sense of this data?
• As they collect data the researcher must ask
– Why do the participants act as they do?
– What does this focus mean?
– What else do I want to know?
– What new ideas have emerged?
– Is this new information?
 Data Analysis After Collection
• One way is to follow three iterative steps
1. Become familiar with the data through
1. Reading
2. Memoing
2. Exam the data in depth to provide detailed
descriptions of the setting, participants, and
activities.
3. Categorizing and coding pieces of data and
grouping them into themes.
Data Analysis After Collection
Summarising
– Reading and memoing
• Read write memos about field notes.
– Describing
• Develop comprehensive descriptions of setting, participants,
etc.
– Classifying
• Breaking data into analytic units.
• Categories
• Themes
 Data interpretation

• Summaries of data  interpretation of results.

– What tools are used for interpretation?
– Logic
– Knowledge of literature and theory.
– Experience.
 Interpreting Findings
• Interpretation is not neutral
• Reflect about the personal meaning of the data
• Compare and contrast personal viewpoints with
the literature
• Address limitations of the study
• Make suggestions for future research
 Data interpretation continued…
• Description of the sample
– Describe the study population by producing tables showing the
distribution of important variables e.g. sex, age, sex by age,
morbidity, nutritional status, nutritional status and age, nutritional
status and sex, nutritional status and morbidity, etc.

• Establish the links and association among the various

variables and the nutritional status
– Statistical analysis could be used to determine links or associations
between various quantitative data.
– Further links between qualitative data and the resulting nutritional
status could be established guided by the conceptual framework.
 Data Interpretation
• Answer these four questions
– What is important in the data?
– Why is it important?
– What can be learned from it?
– So what?
• Remember
– Interpretation depends on the perspective of the
researcher.
• Why?
 Interpretation
• One technique for data interpretation (Wolcott)
– Extend the analysis by raising questions
– Connect findings to personal experiences
– Seek the advice of “critical” friends.
– Contextualize findings in the research
• Converging evidence?
– Turn to theory
 Making comparisons with the
Literature
• Interpret the data in view of past research
• Show how the findings both support and contradict
prior studies
– “These findings are consistent with other studies in regard
to duration. It has been found that the length or duration
of service learning projects has an impact on student
outcomes, with the longer duration projects having greater
impacts. However, significant differences are not found in
projects lasting over 18 weeks (Conrad & Hedin, 1981).
The project on which this study focused was examined
over a year and a half period of time; thus it is considered
to be long in duration which helps to explain its impact on
student outcomes.”
 Synthesis of the data
• Once researchers have a sense of what data
mean and have identified categories and themes
, it is time to start offering some hypothesis or
propositions from the study.
• This may describe the relationships among the
categories that have been identified.
• They may also show how the data fit with the
proposed research problem at the beginning of
the study
 Synthesis of the data
• Revisit some of the techniques, look for
correlation, how well is your problem, research
question and methods are correlated.
• For example, using diagram, a table, or a flow
chart may help researcher communicate the
findings they have come up through analysis
 Synthesis of the data
• The ultimate goal of synthesizing the analysis of
data is to ensure that interpretation are clearly
communicated to the readers of the research
• It is therefore necessary that there is a clear
connection between the data, the categories that
emerge from the coding process and the
interpretations offered.
• It must be clear to the reader how the
researchers drew their conclusions in relation to
the identified study and the collected data
 Reporting the results
• Tables summarize statistical information
• Figures (charts, pictures, drawings) portray
variables and their relationships
• Detailed explanations about statistical results
 Report Findings
• Multiple perspectives for each theme
• Metaphors and analogies
• Quotes
• Detail
• Tensions and contradictions
 Discussing the results
• Summarise major results

• Explain why they occurred

• Advance limitations

• Suggest future research

 “Sci vis” versus “Info vis”

• Visualization: converting raw data to a form

that is viewable and understandable to
humans.

• Scientific visualization: specifically concerned

with data that has a well-defined representation
in 2D or 3D space (e.g., from simulation mesh
or scanner).

*Adapted from The ParaView

Tutorial, Moreland
 Information visualisation

• Information visualization: concerned with data

that does not have a well-defined
representation in 2D or 3D space (i.e., “abstract
data”).
Goals of Information Visualisation

More specifically, visualization should:

– Make large datasets coherent

(Present huge amounts of information compactly)
– Present information from various viewpoints
– Present information at several levels of detail
(from overviews to fine structure)
– Support visual comparisons
– Tell stories about the data
 Two Different Primary Goals:

Explore/Calculate
Analyse
Reason about Information
Communicate
Explain
Make Decisions
Reason about Information
 Why Visualisation?
Use the eye for pattern recognition; people are good at
scanning
recognizing
remembering images

Graphical elements facilitate comparisons via

length
shape
orientation
texture
Animation shows changes across time
Color helps make distinctions
Aesthetics make the process appealing
 Graphs
• A visual display that illustrates one or more
relationships among entities
• A shorthand way to present information
• Allows a trend, pattern, or comparison to be easily
apprehended
• Graphs are easy to read, and highlight distribution’s
shape. The are useful because they show the full
range of variation and identity data anomalies that
might be in need of further study.
• Most common are bar charts, histograms, and
frequency polygon.
 Why Text is Tough

• Text consists of abstract concepts

– which are difficult to visualize
• Text represents similar concepts in many
different ways
– space ship, flying saucer, UFO, figment of imagination

• Text has very high dimensionality

– Tens or hundreds of thousands of features
– Many subsets can be combined together
 Validate the Accuracy of Your
Findings
• Member checking: Members check the
accuracy of the account
• Triangulation: Using corroborating evidence
• External: Hiring the services of an individual
outside the study to review the study
THANK YOU