1

Critique of a Quasi-Experimental Research Study

Olivia Engle, Erin Council, Natalie Dudding, Alexis Eytcheson, and Shelby Furubayashi

Department of Nursing, Walsh University

NURS 702: Advanced Nursing Research

Dr. Shelly Amato-Curran

February 5, 2023
 2

In the research article by Melender et al. (2020), the problem described is that evidence-based

practice (EBP) is often deficient in practice. It is discussed that the varied implementation of evidence-

based practice is due to lack of education regarding evidence-based practice (Melender et al., 2020).

This problem is very relevant to nursing practice as evidence-based practice is something that should

always be utilized by nurses and healthcare professionals to provide the best care and improve

outcomes for patients. In this study, evidence-based practice education was provided, and

questionnaires were handed out before, immediately after, and six months after this education

(Melender et al., 2020). The results of this study will indicate whether implementing EBP competencies

among nursing staff will improve the use of EBP in practice.   

Theoretical framework is a general idea that is the basis for the research being completed

(Kivunja, 2018). The theoretical framework for this article is the pedagogical framework which helps to

clarify different learning and educational methods. In this study there is no conceptual framework

described. In research, the conceptual framework helps to identify the entire plan for the study

including research questions, methods, data collection, interpreting results among others (Kivunja,

2018).  This article has three clearly stated research questions that flow directly from the goal of the

study which helps to clearly indicate the purpose of the study. Questions including does the intervention

impact the use of EBP by participants, does it improve the participants’ skillsets, and what are their

reactions to the intervention (Melender et al., 2020). Based on the research questions and the

introductory information, it is implied that the outcome of the study will show an increase in EBP

knowledge and use after the competencies are completed. 

The literature reviewed in this study evaluated a comprehensive list of studies that are relevant

and recent within the years of 2011 and 2018 on EBP educational interventions for health professionals

(Melender et al., 2020). All the studies reviewed focused on reviewing one of the elements that the
 3

Classification Rubric for EBP assessment Tools in Education (CREATE) believe are prudent in overall EBP

application in healthcare professionals which include, knowledge, skills, self-efficacy, attitudes,

behaviors, and learners’ reactions (Melender et al., 2020). The studies reviewed, assessed EBP

knowledge by one of the logical methods including an objective comprehensive test, or a subjective

voluntary self-evaluation on EBP practices as they were based on reasons and not assumptions (Tappen,

2023). Through the various studies, differing opinions were concluded throughout the six elements

evaluated, prompting an updated study with a new strategy. Building off the related studies and their

findings that focused on the six elements above, the Western Finland Hospital was able to conduct a

study in hopes to strengthen the EBP educational interventions through an objective only approach in

evaluation (Melender et al., 2020). Utilizing these seminal studies that are influential in incorporating

EBP within practice is a solid base for this study to further evaluate with a new method of an objective

evaluation (Tappen, 2023).  

This study followed The Code of Ethics of the World Medical Association guidelines and received

approval from the director of nursing at the associated hospital district (Melender et al., 2020). Nurse

leaders were encouraged to participate but were given the option to opt out due to lack of time to

participate which minimized risk to study participants. Participants were given incentives such as

continuing education credits that were provided by the hospital for completing the EBP education.

Informed consent was obtained by all participants and their identities remained confidential to protect

privacy (Melender et al., 2020).

This study is a quasi-experimental design which employs all the elements of an elemental design

besides randomization (Tappen, 2023). Study groups had a pretest, an intervention via EBP education,

an immediate posttest, and a posttest after six months (Melender et al., 2020). The content of the

intervention was conformed to competencies on a bachelor’s level of education (Melender et al., 2020).
 4

The study gives enough information to replicate the study as it is based solely on one primary group,

with a self-administered questionnaire developed by the researchers, based on education interventions

implemented (Melender et al., 2020). This study may be difficult to replicate because EBP is constantly

evolving.   

            Nurse leaders were invited to participate in this study, but if they could not participate, another

nurse on the unit could take their place leaving room for the target population to change based on

interest (Melender et al., 2020). The interest in the study and differentiation of the participants would

be considered a threat to internal validity based on selection bias (Tappen, 2023). There was a focus on

nursing leadership, but other healthcare professionals and social workers also had equal opportunity to

participate in the study, so the sample preferences were expanded (Melender et al., 2020). The chosen

sampling is effective for the type of study chosen as all the participants were exposed to the evidence-

based practice education and the same questionnaire. The participants were split into two rounds, the

first round with 32 participants and the second with 51 (Melender et al., 2020). This number of

participants was sufficient for the research as the number of the actual sample was similar to the

amount the study was created for and had invited for the research.  

The instrument used in this study was a questionnaire developed using the CREATE framework

and was necessary to develop due to existing tools being inadequate for this study’s goals (Melender et

al., 2020). The theoretical framework of this study was the pedagogic theory that focuses on the values

and principles that influence our approaches to learning, teaching, and assessment (Information Literacy

Group, 2023). The instrument used fits with the pedagogic theory as it determines how effective

education on EBP is. The data collection via self-administered questionnaire was realistic due to a low

number of study participants. Prior to the developed instrument being used in this study, it was

pretested with a group of master’s students enrolled in a Development and Management of Social and
 5

Health-Care Services program which clarified some wording in the instrument before approval

(Melender et al., 2020). 

This quasi-experimental study uses quantitative analysis with statistical measures to answer the

question asked by the study. Statistical analysis is used to collect a wide range of information over a

period, and it helps make sense of the numbers in the study and gives them purpose (Ali & Bhaskar,

2016). The analysis was done with the use of SPSS for Windows, release 23.0 (Melender et al., 2020).

Feedback variables and other variables were statistically analyzed to provide answers regarding the

“data, frequency, and percentages” regarding how the participants were affected (Melender et al.,

2020).  The study question is how healthcare professional’s knowledge, attitude, self-efficiency, and

attitude has changed after being educated on evidence-based practice. Quasi-experimental design is a

type of quantitative design which uses statistical analysis in this study, they all work together to collect,

organize, and give meaning to the evidence yielded during the study (Harris et al., 2006).  

The measurement used in this study is appropriate for the type of statistics. This study resorted

to the use of non-parametric tests because they were not able to normally distribute observations at all

points in time (Melender et al., 2020). It did still allow them to compare repeated measurements and

find discrepancies (Melender et al., 2020). The statistical results are presented clearly in both text and in

numerical presentation. The tables given are easy to read and labeled with the appropriate information.

The in-text explanation of the test mentioned in the tables are well defined and explanation of the

measurements from 1-3 are made. The tables are simple, and their accuracy is viable. 

Researchers attempted to control threats to internal validity by designing the educational

intervention used toward EBP competencies on a bachelor’s level and included all steps of EBP.  This

strengthened the intervention because after their review of literature, it was found that EBP educational

interventions did not always include all steps of EBP.  In this study, the Guideline for Reporting Evidence-
 6

based Practice Educational Interventions and Teaching checklist was utilized which provides framework

on EBP documentation. It was a limitation that a new instrument was used for data collection in this

study.  However, using the CREATE framework and pretesting improved the validity of the instrument.  

An internal threat to validity is a factor that may interfere with the interpretation of results in an

experiment (Tappen, 2023).  One internal threat to validity in this study is the testing method.  When

the same questions are used on a pretest and posttest, how well the participants do on the posttest will

be affected (Tappen, 2023).   Another large threat to this study’s internal validity is attrition which

occurs when participants drop out of the study (Tappen, 2023).  At the start of the study, there were 83

participants and at the six-month follow-up, there were only 48 participants which may have led to

some participant bias (Melender et al., 2020). This study is a quasi-experimental design, and no control

group was used which also threatened internal validity. 

An external threat to validity is a factor that may affect how well a study can be generalized to

others (Tappen, 2023).  To positively impact external validity for this study, the population of

participants could have been more randomly selected. Instead of focusing on just nursing leadership,

the invite could have included all staff nurses ultimately helping to generalize results.  Also, the small

sample size of the study creates a threat to external validity (Tappen, 2023). Increasing the sample size

could have contributed to external validity for this study. 

One limitation in this study is that there cannot be comparison between groups due to only one

group being evaluated. Because EBP is constantly evolving, this study would be difficult to repeat

(Melender et al. 2020). The CREATE framework was used to increase the competency and validity of this

study. The results of this study show that educational interventions were successful in improving EBP

use in practice. This study is generalizable to healthcare staff as EBP techniques have validity and have

been studied extensively since Florence Nightingale in the 1800s (Mackey & Bassendowski, 2017).
 7

References

Ali, Z., & Bhaskar, S. B. (2016). Basic statistical tools in research and data analysis. Indian journal of

anesthesia, 60(9), 662–669. https://doi.org/10.4103/0019-5049.190623

Harris, A. D., McGregor, J. C., Perencevich, E. N., Furuno, J. P., Zhu, J., Peterson, D.E., & Finkelstein, J.

(2006). The use and interpretation of quasi-experimental studies in medical informatics. Journal

of the American Medical Informatics Association: JAMIA, 13(1), 16–23.

https://doi.org/10.1197/jamia.M1749

Howard, B., Diug, B., & Ilic, D. (2022). Methods of teaching evidence-based practice: A

systematic review. BMC Medical Education, 22(1). https://doi.org/10.1186 /s12909-022-03812-x

Information Literacy Group. (2023). Pedagogic theory. Information literacy group.

https://infolit.org.uk/teaching/developing-your-teaching/pedagogic-theory/

Kivunja, C. (2018). Distinguishing between theory, theoretical framework, and conceptual framework: A

systematic review of lessons from the field. International Journal of Higher Education, 7(6), 44-

53.

Mackey, A., & Bassendowski, S. (2017). The History of Evidence-Based Practice in

Nursing Education and Practice. Journal of professional nursing: Official Journal of the

American Association of Colleges of Nursing, 33(1), 51–55.

https://doi.org/10.1016/j.profnurs.2016.05.009

Melender, H.-L., Salmela, S., & Pape, B. (2020). A quasi-experimental study of a basics of evidence-based

practice educational intervention for Health and Social Care Professionals. SAGE Open Nursing,

6, 237796082092595. https://doi.org/10.1177/2377960820925959 
 8

Tappen, R. (2023). Advanced nursing research: From theory to practice. (3rd ed). Jones & Bartlett

Learning.