Critical appraisal checklist for Randomized Controlled Trials (RCT)

Adapted from JBI WEBSITE: HTTPS://JOANNABRIGGS.ORG/ -CRITICAL APPRAISAL TOOLS: HTTPS://JOANNABRIGGS.ORG/CRITICAL-

APPRAISAL-TOOLS

Article Title: The effect of aquatic exercise training on heart rate variability in patients with
coronary artery disease

Reviewer: Klein, Kennedy

Yes No Unclear NA
1. Was true randomization used for assignment of x
participants to treatment groups?
2. Was allocation to treatment groups concealed? x

3. Were treatment groups similar at the baseline? x

4. Were participants blind to treatment assignment? x

5. Were those delivering treatment blind to treatment x

assignment?
6. Were outcomes assessors blind to treatment x
assignment?
7. Were treatment groups treated identically other than x
the intervention of interest?
8. Was follow up complete and if not, were differences x
between groups in terms of their follow up adequately
9. Were participants analyzed in the groups to which they x
were randomized?
10. Were outcomes measured in the same way for x
treatment groups?
11. Were the instruments used to measure outcomes x
reliable and valid?
12. Was appropriate statistical analysis used? x

13. Was the trial design appropriate, and any deviations x

from the standard RCT design (individual
randomization, parallel groups) accounted for in the
conduct and analysis of the trial?

Overall credibility of article results per your assessment on a scale of 0-10, with 0 –“I don’t trust
the results, as the intervention study outcomes are questionable” to 10 – “I will definitely use the
results of the study in planning interventions for my patients” 7/10. This article did disclose that a
limitation of the study was the water-based group- was significantly older than the other 2 groups.
They also mentioned that a limitation was this study cannot be extended to those with other
cardiovascular conditions; it is strictly for those with CAD.

© JBI, 2020. All rights reserved. JBI grants use of these tools for research purposes only. All other enquiries
should be sent to jbisynthesis@adelaide.edu.au.
Explanation of critical appraisal of Randomized Controlled Trial (RCT)

Brief and structured summary of the article in a form of Abstract

The question this study was examining was the effect of aquatic exercise training on heart
rate variability in patients with coronary artery disease. The study design was a prospective,
randomized, open-label parallel trial. Participants were randomly placed into one of three groups:
water-based, land-based, or control group. Both the water-based and land-based groups
participated in a fourteen-day trial that included two 30-minute exercise sessions. The water-
based training included one water aerobics session of walking, side-stepping, arm cycling, etc., and
one calisthenics session of triceps extensions, triceps dips, leg press, etc. The land-based training
consisted of one aerobic session using a bicycle ergometer, and one calisthenics session.
Participants were tested, using a twenty-minute 12-channel high-resolution electrocardiogram
(ECG) recording, prior to the implementation of interventions and after. The study observed a
significant decrease in both low-frequency and high-frequency indices for the water-based group
(p=0.036). They also observed a significant increase in the non-linear heart rate variability a2
parameter in the land-based group (p=0.003) and a significant increase in the non-linear heart rate
variability a1 parameter in the water-based group (p=0.043). This study shows that aquatic
exercise training is a safe and effective form of training for individuals with coronary artery
disease. The results of the study or only applicable to those with coronary artery disease. This
study does not apply to those with other cardiovascular issues such as heart failure.

1. Was true randomization used for assignment of participants to treatment groups?

True randomization was used for the assignment of participants to the water-based, land-
based, or control group. The article said they used the urn-randomized method to place the
participants in groups. This method has been proven to eliminate experimental bias and
achieve balance in treatment assignments and complete randomization. 1 The study placed
thirty participants in each group and conservatively estimated a 10-20% drop-out rate. 2

2. Was allocation to groups concealed?

Allocation to groups was concealed. As mentioned above, the study used an urn-
randomization method in which the use of sealed envelopes were used. The sealed envelopes
were distributed and placed patients into their assigned groups. This method eliminates the
risk of researchers purposefully intervening and rearranging assignments for the study’s
benefit.

3. Were treatment groups similar at the baseline?

Treatment groups were similar at baseline in terms of the diagnosed condition. The study
included those with “coronary artery disease after a recent myocardial infarction and/or
revascularization procedure undergoing stationary cardiac rehabilitation”. 2 The study provided
a long list of exclusion criteria which included conditions such as uncontrolled hypertension,
uncontrolled/high-risk dysrhythmias, contraindications to exercise, presence of a permanent
pacemaker, over the age of eighty, etc..2
© JBI, 2020. All rights reserved. JBI grants use of these tools for research purposes only. All other enquiries
should be sent to jbisynthesis@adelaide.edu.au.
4. Were participants masked (or blinded) to treatment assignment?
Participants were not blinded to treatment assignment. Participants were made aware of what
group they were assigned to and what would be asked of them. It would be hard to blind
participants in this study because the participant is assigned to either water-based, land-based,
or control group. The study did not mention whether participants had a change in behavior
when they discovered which group they would be assigned to. This may be something for the
researchers to address as it could lead to those placed in the control group believing they may
not receive benefit from the study compared to those undergoing interventions.

5. Were those delivering treatment blind to treatment assignment?

It was unclear whether those delivering the treatment were blind to the treatment
assignment. There was no information given about those delivering treatment or if they were
qualified. The study only mentions what each intervention group had to do. For example, in
the water-based group, the study mentioned treatment included one water aerobics session
and one calisthenics session. It did not mention how those sessions were delivered.

6. Were outcomes assessors blind to treatment assignment?

It was unclear if outcomes assessors were blind to treatment assignment. There is no
information in the article mentioning outcomes assessors. This should be made clear in the
article to eliminate any question of bias among researchers.

7. Were treatment groups treated identically other than the intervention of interest?
Intervention groups were treated identically when it came to physical exercise, but the study
did mention that “medicines were adjusted when necessary to ensure that blood pressure and
heart rate were optimally controlled”.2(p6) The article also mentioned a “major limitation to the
study” was the control group was made aware of the benefits of regular exercise and while not
having undergone supervised training, they may have participated in unsupervised exercise
that was not accounted for.2

8. Was follow-up complete and if not, were differences between groups in terms of their
follow-up adequately described and analyzed?
Follow-up was completed at the end of the fourteen-day intervention period. Just like before
the fourteen days, after the fourteen days all participants “underwent a twenty-minute high-
resolution ECG recording”.2(p10) The ECG recording was the measure used to compare the three
groups. There was no mention of long-term follow-up.

9. Were participants analyzed in the groups to which they were randomized?

Participants were analyzed in the groups they were originally assigned. Researchers accounted
for a 10-20% dropout rate; there was one participant who dropped out due to an upper
© JBI, 2020. All rights reserved. JBI grants use of these tools for research purposes only. All other enquiries
should be sent to jbisynthesis@adelaide.edu.au.
 respiratory tract infection. The participant was part of the water-based group and researchers
did not mention if this had an impact on the data collected. The article did mention that a
sample size of twenty-six participants in each group is sufficient enough to determine
significance among groups.2

10. Were outcomes measured in the same way for treatment groups?
Outcomes were measured in the same way in all three groups. All participants underwent the
ECG testing as stated before, and the researchers made sure to take the measurements at the
same time, between 9 AM and 11 Am, and under the same conditions, ten minutes after
resting in a quiet room that was kept at 22-24.2 All participants were given the same
instructions prior to arriving for testing.

11. Were the instruments used to measure outcomes reliable and valid?
The study did not explicitly say whether the instruments used to measure outcomes were
reliable and valid. The study did, however, describe how the ECG recordings were taken and
how it was the same for each participant. They also listed and described all software that was
used to analyze the data. It did not mention how many researchers collected the ECG
recordings or if they had done so prior to this study. The article did mention that “After
automated editing, an experienced observer reviewed and corrected all tracing manually”. 2(p11)
This sentence does imply that the data collected is reliable and checked. This may raise the
question of the observer tampering with the data, however, the article mentioned he only
eliminated abnormal beats.

12. Was appropriate statistical analysis used?

There was one independent variable with three levels: water, land, or control, and two
dependent variables: R-R interval and heart rate variability (HRV) parameters. R-R interval was
measured using the Schiller SEMA-220 where each beat was classified as normal, ventricular
ectopic, supraventricular ectopic, or unknown. The HRV parameters included a linear HRV
analysis, frequency-domain linear HRV measure, and non-linear HRV analysis. For data
distributed normally, it was presented as mean (standard deviation) and for non-normal
distribution, it was presented as median (interquartile range). 2 Between-group differences
were tested using ANOVA and differences from baseline within each group were measured
using a paired sample t-test for normal distribution. 2 These are appropriate statistical
procedures for this type of study. The effect size was calculated at twenty-six participants per
group needed to see statistical significance; the study used thirty per group. The breakdown of
how the analysis was performed was explained well.

13. Was the trial design appropriate for the topic, and any deviations from the standard RCT
design accounted for in the conduct and analysis?

© JBI, 2020. All rights reserved. JBI grants use of these tools for research purposes only. All other enquiries
should be sent to jbisynthesis@adelaide.edu.au.
 A randomized control trial was appropriate for this type of research question with no
deviations from a standard RCT. Baseline characteristics were made available with only one
between-group difference mentioned. This is the water-based group being significantly older.
All dependent variables were compared to the baseline and given in table format to easily see
variability. The results of this study are meaningful to clinical practice to help those with
coronary artery disease improve cardiovascular health. In my opinion, investment in the trial is
worth it. This trial can be tweaked to observe many other cardiovascular conditions.

Additional consideration

The article is published in the Journal of Cardiovascular Development and Disease which is a
very credible journal. For the main author of this article, this is not his first publication. His
knowledge in the field is extensive with ten published articles to his name. His group of co-
authors are also strong in the field of study and are credible in their work. The results of the
study are consistent with other studies done comparing exercise and cardiovascular health in
those with coronary artery disease. The article mentioned this study is the first to study HRV,
through different modes of exercise, in patients after a coronary artery disease event.

Why you should or should not use this evidence?

The results of the study were backed by evidence and sound theory. Exercise has been proven
in the past to improve cardiovascular health and this study showed no difference. Both the
water-based training and land-based training participants were monitored closely to ensure
the safety of each participant and to ensure intervention procedures were carried out
properly. I would use the results of this study for my own patients because the risk of injury
during this study was minimal and the effects are beneficial to not only those with CAD, but to
all. This was a short, fourteen-day intervention allowing it to be more practical to those who
maybe don’t have the time to be part of a long-term study. This study can also act as an
introduction to exercise and teach participants different ways to exercise in a safe manner
while not under supervision. The article scientifically proves that even someone with CAD can
see the benefit of exercise in as little as fourteen days.

© JBI, 2020. All rights reserved. JBI grants use of these tools for research purposes only. All other enquiries
should be sent to jbisynthesis@adelaide.edu.au.
References

1. Wei LJ, Lachin JM. Properties of the urn randomization in clinical trials. Control Clin Trials.
1988;9(4):345-364. doi:10.1016/0197-2456(88)90048-7
2. Jug B, Vasić D, Novaković M, Avbelj V, Rupert L, Kšela J. The effect of aquatic exercise
training on heart rate variability in patients with coronary artery disease. J Cardiovas Dev
Dis. 2022; 9(8):251. https://doi.org/10.3390/jcdd9080251.

© JBI, 2020. All rights reserved. JBI grants use of these tools for research purposes only. All other enquiries
should be sent to jbisynthesis@adelaide.edu.au.