15 LEARNING UNIT 15

15 MIXED METHODS AND EVALUATION

RESEARCH

15.1 INTRODUCTION
The previous learning units focused on quantitative and qualitative research,
respectively. In recent research studies, the use of a combination of qualitative
and quantitative research was used in order to utilise the best of both worlds.
Learning unit 15 is intended to help you understand the important concepts of mixed
method research. Chapter 20 of the prescribed book (Fouché, C.B., Strydom, H. &
Roestenburg, W.J.H. [Eds.]. 2020. Research at grass roots – for the social sciences and human
services professions. 5th edition. Cape Town: Van Schaik – ISBN 9780627038211) is
titled mixed method research. In this chapter, the main elements that play a role
in professional practice and research ethics are briefly discussed. This learning unit
also includes chapter 21 of the prescribed textbook, titled evaluation research.

FIGURE 15.1
Module outline (lesson 12)

SMA4801/1105


15.2 LEARNING OUTCOMES

After working through this learning unit and the relevant parts of the prescribed
book, you should be able to:

• define the concepts of multimethod, mixed methods and mixed research approaches
• explain the nature of mixed methods research
• trace the evolution of mixed methods research
• demonstrate an understanding of the appropriate steps in designing a mixed
methods research study
• distinguish between the different core mixed methods research designs
• articulate the different conceptual levels of evaluation research
• demonstrate an understanding of the integrated process of evaluation
• become better acquainted with the purposes and types of evaluation research
• articulate how the types of evaluation are linked to the programme logic and
life cycle
• identify appropriate research designs and methods in evaluation research

15.3 CONTENT OF THE LEARNING UNIT

To achieve the aim and learning outcomes of this unit, you will cover the following
topics in chapter 20 of the prescribed book:

(1) characteristics of mixed methods research

(2) evolution of mixed methods research
(3) mixed methods research process

We will also look at the following topics in chapter 21 of the prescribed book:

(1) characteristics of evaluation research

(2) a process model for evaluation research
(3) determine the approach to the evaluation
(4) conceptualise the programme logic
(5) decide on the purpose of the evaluation
(6) design the evaluation
(7) select the evaluation methods
(8) making sense of it all

After reading and thinking about the contents of chapter 20 and chapter 21 of
the prescribed book, you will have to explain the concepts of mixed methods and
evaluation research approaches.

15.4 KEY CONCEPTS

The following key terms are important in this learning unit:

• convergent design • evaluation research

• explanatory sequential design • formative
• exploratory sequential design • impact
• mixed methods research • life cycle
• mixed research approaches • methods
• multimethod research approach • needs assessment

106
 LEARNING UNIT 15: Mixed methods and evaluation research

• challenges • outcomes
• design • process evaluation
• evaluability assessment • programme logic
• evaluation process • purpose; summative
• programme monitoring

Remember to take a look at the glossary of terms that has been uploaded on myUnisa
under Additional Resources. If you see a term and you do not know what it means,
stop and go to the glossary. This will familiarise you with the terminology of research
and you will get comfortable with the content of this module much quicker.

https://mymodules.dtls.unisa.ac.za/pluginfile.php/15594258/mod_folder/content/0/
SMA4801_Glossary%20of%20terms%20Eng.docx?forcedownload=1

After working through the learning unit, you should be able to explain all of the
above-mentioned concepts. These concepts are all explained or defined in the
prescribed textbook and this unit.

15.5 CHARACTERISTICS OF MIXED METHODS RESEARCH

Study the section titled characteristics of mixed methods research in the prescribed
textbook. Mixed methods research involves the collection, analysis and integration
of both qualitative and quantitative research in different phases of a research study.
The advantages of using mixed methods include

• completeness
• complementary
• resolving complex issues
• triangulation
• enrichment
• flexibility
• quality results
• guiding the development of measuring instruments

Take a look at this video that will help you to distinguish between qualitative,
quantitative and mixed methods research:

https://www.youtube.com/watch?v=hECPeKv5tPM

ACTIVITY 15.1: C
 HARACTERISTICS OF MIXED METHODS
RESEARCH

What differences are there between multimethod research, mixed methods research
and mixed research approaches?
Various authors differ in their views of these concepts, but mixed methods investigations
generally involve integrating quantitative and qualitative data collection and analysis into
a single study or a programme of inquiry, while carefully considering the implications
of such “mixing” for theoretical and philosophical underpinnings. This form of research
is more comprehensive than multimethod research, which consists of multiple forms of
qualitative data and/or quantitative data collection. Mixed methods research also has
a close resemblance to mixed research approaches, which include all phases of the
research process, such as philosophical worldviews, methods and the interpretation
of results.

SMA4801/1107


15.6 EVOLUTION OF MIXED METHODS RESEARCH

Study the section titled evolution of mixed methods research in the prescribed
textbook. This section of the prescribed textbook provides an overview of the
formative period (1950s to 1980s), the paradigm debate period (1970s to 1990s),
procedural development (1990s to 2000), and the reflection and refinement period
(2000 to present).

15.7 MIXED METHODS RESEARCH PROCESS

Study the section titled mixed methods research process in the prescribed textbook.
This section of the prescribed textbook explains how a researcher can determine
whether mixed methods research is the best research approach. It also provides
information on the philosophical and theoretical foundations of mixed methods
studies. It also provides information on the core elements of mixed methods design.
The key considerations when choosing a mixed methods design for a research study
is summarised in figure 15.2.

FIGURE 15.2
Key considerations of mixed methods research design

ACTIVITY 15.2: MIXED METHODS RESEARCH PROCESS

What value is added to studies through mixed methods research?

• Completeness: Mixed methods research enables you to simultaneously address

a range of confirmatory and exploratory questions with both the qualitative and
quantitative approaches and therefore verify and generate theory in the same study.
• Complementarity: Mixed methods research provides strengths that offset the
weaknesses of both quantitative and qualitative research, and therefore has the
potential to provide better (stronger) inferences.

108
 LEARNING UNIT 15: Mixed methods and evaluation research

• Resolution of complex issues: Mixed methods research provides more comprehensive

evidence for studying a complex research problem than either quantitative or
qualitative research alone.
• Triangulation: Mixed methods research encourages you to collaborate across the
occasional antagonistic relationship between quantitative and qualitative researchers.
If findings are corroborated across different approaches, then greater confidence
can be held in the singular conclusion; if the findings conflict, then you had greater
knowledge and can modify interpretations and conclusions accordingly. In many
cases, however, the goal of mixing is not to search for corroboration but rather to
expand your own understanding.
• Enrichment: Mixed methods research provides the opportunity for a greater
assortment of divergent views and perspectives and alerts you to the possibility
that issues are more multifaceted than they may have initially supposed.
• Flexibility: Mixed methods research is “practical” in the sense that you are free to
use different methods to address a research problem and can combine inductive
and deductive reasoning processes.
• Results quality: Mixed methods research explains the true nature of a phenomenon
under investigation and increases the quality of the findings.
• Guide development of measuring instruments: Mixed methods research enriches
the development of appropriate tools based on the findings from either of the
approaches (e.g. interviews can inform survey design, or survey results can inform
interview schedules).

When selecting a specific mixed methods design, the research can use a convergent
design (figure 15.3), an explanatory sequential design (figure 15.4) or an exploratory
sequential mixed methods design (figure 15.5).

FIGURE 15.3
Convergent design of mixed methods research (Fouché et al., 2022:430)

SMA4801/1109


FIGURE 15.4
Explanatory sequential design of mixed methods research (Fouché et al.,
2022:431)

FIGURE 15.5
Exploratory sequential mixed methods design (Fouché et al., 2022:433)

110
 LEARNING UNIT 15: Mixed methods and evaluation research

ACTIVITY 15.3: MIXED METHODS RESEARCH PROCESS

What is the value of a uniform notational system in describing the different types of
mixed methods research design?
The ways in which a researcher might combine quantitative and qualitative methods are
almost limitless, restricted only by the research question. Mixed methods researchers
tend to use notational conventions to describe how the timing, weighting and mixing
decisions contribute to the overall design. The basic notational system uses plus (+)
symbols and arrows (→), as well as uppercase and lowercase letters. A plus sign indicates
that quantitative and qualitative data are collected concurrently (at the same time) and
an arrow indicates that they are collected sequentially (one followed by the other). The
use of uppercase letters indicates higher priority or weighting for a particular method
(i.e., QUAN or QUAL), while lowercase letters indicate lower weighting or priority (i.e.,
quan or qual).

Take a look at the video below for a discussion of the use of technology in mixed
methods research:

https://www.youtube.com/watch?v=4vhXUJQecVA

15.8 CHARACTERISTICS OF EVALUATION RESEARCH

Study the section titled characteristics of evaluation research in the prescribed
textbook, paying close attention to the discussions of evaluation research and what
it entails.

ACTIVITY 15.4: EVALUATION RESEARCH

How would you define evaluation research?

Evaluation research is defined as the process of collecting credible data and using it to
make judgements about the worth of a product, programme, service or process. In its
simplest form, evaluation is the process of discovering the value of something.

ACTIVITY 15.5: EVALUATION RESEARCH

What are the inherent challenges of evaluation research?

(1) Evaluators can be faced with demands for evidence that must be balanced with an
increasingly sophisticated understanding of what social settings, cultural values
and influential evaluations actually entail. This includes the engagement required
to ensure indigenous groups are properly represented in an evaluation.
(2) Advantages of getting close to practice must be balanced against the need for
objective measures to demonstrate outcomes.
(3) In complex social contexts, researchers often face wicked problems. It can be dif-
ficult in such circumstances to decide what a valid outcome is and the standards
it should be measured against.
(4) Researchers have to balance the context-specific nature of evaluative research
with the need for generalisable knowledge – especially where the intention of the
evaluation is to upscale service delivery or policy design.

SMA4801/1111


(5) In practice contexts where often very limited resources are directed at service
delivery and research capability is limited, the availability of data about programme
delivery and outcome measures can be weak or absent.
(6) Researchers have to manage unintended findings or opposing findings that may
threaten the existence of a service, programme or intervention and the careers of
the people involved in the programme under consideration.

15.9 A PROCESS MODEL FOR EVALUATION RESEARCH

Study the section titled a process model for evaluation research in the prescribed
textbook. Every evaluation is unique but still involves a measure of the scientific
research process. Figure 15.6 below provides a summary of the main elements of a
research process for evaluation research.

FIGURE 15.6
Integrated process model for evaluation research (Fouché et al., 2022:440)

15.10 DETERMINE THE APPROACH TO THE EVALUATION

Study the section titled determine the approach to the evaluation in the prescribed
textbook. This section of the prescribed textbook provides more information on the
philosophical orientation to evaluation research.

15.11 CONCEPTUALISE THE PROGRAMME LOGIC

Study the section titled conceptualise the programme logic in the prescribed
textbook. This section of the prescribed textbook provides more information on
the programme logic, which sets out what a project will do and how it will be done.
Figure 15.7 provides an illustration of what such programme logic could look like.

112
 LEARNING UNIT 15: Mixed methods and evaluation research

FIGURE 15.7
Graphic example of programme logic (Fouché, 2022:443)

15.12 DECIDE ON THE PURPOSE OF THE EVALUATION

Study the section titled decide on the purpose of the evaluation in the prescribed
textbook. This section of the prescribed textbook makes a distinction between
formative evaluation – needs assessment – and a formative evaluation – evaluability
assessment. Process evaluation involves programme monitoring. Additionally, the
chapter provides information on the summative evaluation of the impact or outcome
of a service, programme or intervention. Another form of summative evaluation is
an efficiency analysis which can involve a cost-benefit analysis or a cost-effectiveness
analysis.

15.13 DESIGN THE EVALUATION

Study the section titled design the evaluation in the prescribed textbook. Pay close
attention to the distinction between evaluation approaches, evaluation purposes,
evaluation designs and evaluation methods in this section of the prescribed textbook.
Evaluation research lends itself well to a qualitative research design since it involves
a depicted process, based on experiences of different people that can be captured in
their own words. The perceptions of participants are key in successfully completing
evaluation research.

Watch this video for more information on what evaluation research is:

https://www.youtube.com/watch?v=-UbWNKhQihE

15.14 SELECT THE EVALUATION METHODS

Study the section titled select the evaluation methods in the prescribed textbook.
This section of the prescribed textbook provides more information on community
forums and key informants as the most pertinent methods of evaluation research.

15.15 MAKING SENSE OF IT ALL

Study the section titled making sense of it all in the prescribed textbook. The
researcher has to organise, analyse and interpret the data he/she collected before
it can be reported on in line with the original questions that sparked the research
project. Results cannot just be taken at face value but should be properly analysed
according to the guidelines of the selected research method.

SMA4801/1113


15.16 SUMMARY OF LEARNING UNIT

This learning unit provided an overview of the characteristics of mixed methods
research, the evolution of mixed methods research and the mixed methods research
process. The second part of this learning unit provided more information and guidance
on evaluation research. This included a discussion of the characteristics of and process
model for evaluation research. Researchers are required to determine and document
their approach to the evaluation, prior to conceptualising the programme logic and
deciding on the purpose of the evaluation. The researcher is also responsible for
designing the evaluation, selecting the evaluation methods and properly reporting
on the findings of the evaluation research study.

The next learning unit will look at intervention and participatory research
(Learning unit 16) as part of lesson 13.

15.17 REFERENCES
Fouché, C.B., Strydom, H. & Roestenburg, W.J.H. (Eds.). 2021. Research at grass
roots – for the social sciences and human services professions. 5th edition. Cape Town:
Van Schaik.

Assessment 5 Final portfolio

114