EIPCD Irene Lahde s2293048

Target article: Comparative efficacy of psychological therapies for improving mental

health and daily functioning in irritable bowel syndrome (IBS): A systematic

review and meta-analysis (Laird, Tanner-Smith, Russell, Hollon, & Walker, 2017)

1. Objectives

The authors’ main objectives were to investigate the efficacy of psychological therapies for

improving mental health and daily functioning in adults with IBS, and whether efficacy was

moderated by modality. Additionally, they aimed to explore potential moderators relating to

therapy, trial and participant characteristics.

2. Inclusion/Exclusion Criteria

There was a comprehensive list of inclusion/exclusion criteria.

The authors excluded studies using non-validated mental health measures, but

included non-validated measures of daily functioning, giving the rationale that validated

measures of daily functioning have “only recently become available”. They mentioned SF-36

and Role Physical Scale as measures with “good psychometric properties” but did not

provide details of other scales. According to Ballou and Keefer (2017) there are no available

validated measure of daily functioning for IBS patients. As such, any conclusions by the

authors regarding daily functioning may be weaker than those regarding mental health.

For studies evaluating group therapies, cluster randomization was deemed acceptable.

However, according to Donner and Klar (2004) cluster randomization trials have reduced

statistical efficiency, and potential ethical issues relating to consent. The authors do not

discuss this limitation, and the article does not state which of the eight studies using group

formats used cluster randomization. On further examination of the eight individual studies

with group therapies, only Schröder et al (2012) used cluster randomization.

1
EIPCD Irene Lahde s2293048

3. Literature Search

The authors searched several databases and went through the reference lists of three meta-

analyses. They contacted authors in cases of suspected selective reporting or insufficient

data. However, in order to make the study more replicable, the authors could have included

the search terms in the article, given that the search terms in the Appendix are more diverse

(“yoga”, “tai chi”) than stated in Methods (“terms related to IBS, psychotherapy, and

controlled trials”).

The authors attempted to minimise publication bias by including unpublished studies

in their search. Their rationale was the lack of evidence that unpublished trials are of a

poorer quality. The Appendix stated that out of 831 studies, 48 were unpublished, and

judging from the lack of unpublished studies in the reference list, all 48 were excluded. No

information about them was provided. One possible reason runs counter to the authors’ logic:

are published trials higher-quality after all? Another possibility is that there simply were not

enough unpublished studies in the search results, suggesting that the search was not

extensive enough. They used ProQuest, but they could also have used EthOS, WorldCat, or

OpenThesis.

The authors did not address the limitation of searching exclusively English-language

studies. Egger et al (1997) found that clinical trials are more likely to be reported in an

English-language journal if they contain significant results, whereas non-significant results

are more likely to be published in national journals. Although language bias may have

reduced recently given the shift towards publication in English (Higgins & Green, 2011), this

trend may not be applicable here, given that the meta-analysis included studies from as early

as 1991. Either the authors could have restricted their search to more recent studies or

included studies in other languages.

2
EIPCD Irene Lahde s2293048

4. Screening

The authors said they followed PRISMA guidelines; however, these recommend that the

screening protocol should be stated in Methods. This was only included in the Appendix,

where it was apparent that the first author conducted the whole eligibility evaluation process.

Two other authors were only consulted in unclear cases; however, no information was

provided about these cases.

The authors could have improved their protocol by following Cochrane guidelines

(Higgins & Green, 2011): having more than one author repeat the process to reduce the

possibility that relevant reports could be missed, as well as having more than one author to

undertake the final selection, including a non-expert (to reduce bias related to pre-existing

knowledge/opinions), and blinded to information about the study. The Appendix stated that

all authors were aware of study authors, results, and publication information. This could have

resulted in bias towards certain journals/authors/institutions/results, or towards the 48

unpublished studies. Given the lack of declaration of interests or funding information, this

could be problematic.

5. Data Collection/Extraction

The Appendix stated that a single reviewer entered and coded all data (effect size data

checked by another author): this was the same person who did the screening. This could have

biased the selection process, which could have been improved by having blinded authors, or

multiple authors providing inter-rater reliability.

The coding of modality constitutes one of the biggest weaknesses of the meta-

analysis. The authors admitted that many studies used interventions combining different

treatment types and that this made categorizing them difficult. As such, coding could have

3
EIPCD Irene Lahde s2293048

been undertaken by several authors. The authors neither provided any detail relating to how

they coded modality, nor did they cite studies that were particularly problematic. However,

looking at the individual studies, it is possible that the authors may have underestimated the

extent of the problem. For example, the study by Labus et al (2013) was categorized as

“CBT”, but was in fact a combination of relaxation training with psychoeducation on

maladaptive thinking. Homework was relaxation and symptom monitoring, with no

cognitive/behavioural components. The study by Moss-Morris et al (2010) involved CBT

and relaxation training. The psychodynamic therapy by Guthrie et al (1991) also included

relaxation. Two of the “CBT” studies used mindfulness-based stress-reduction (MBSR)

protocols (Gaylord et al, 2011; Zernicke et al, 2013). The effects of MBSR have been

compared against those of CBT in many studies, suggesting these treatments are distinct (e.g.

Arch & Ayers, 2013; Cherkin et al, 2016). In the discussion, the authors implied that

mindfulness is a type of CBT, but that it was “impossible” to investigate differences between

“different types of CBT”.

6. Statistics

There was no information about who did the statistical analysis. The authors used the

Comprehensive Meta-Analysis Software to pool mean effect sizes and calculate 95%

confidence intervals. They calculated standardized mean differences: the difference between

the two groups’ mean change scores divided by the pooled standard deviation. They cited

recommendations by Higgins et al (2003) as rationale to use random-effects models a priori,

given that the RCTs used different measures, and therefore a common effect size across

studies could not be assumed. Given the differences between studies, the authors followed

good practice (Higgins et al, 2003) and calculated heterogeneity. Given than some studies

had more than one control/treatment arm, they conducted sensitivity analyses to account for

4
EIPCD Irene Lahde s2293048

their decision to exclude one of these arms. They conducted moderator analyses: between-

study variances were computed separately for subgroups, not pooled across.

7. Results, Conclusions, Limitations

Descriptive statistics were only summarized in a table, not in the text. Data on type of control

condition was missing. Nevertheless, they helpfully provided forest plots for results relating

to their primary objectives.

As the results of Egger’s test and the rank correlation test indicated, the authors did

not find evidence of publication bias. They followed Borenstein et al’s (2011) good practice

of including funnel plots (showing no asymmetry), albeit only in the Appendix.

Although the authors provided a table with results from the Cochrane risk of bias

assessment, criteria and details were only included in the Appendix. They discussed that there

were no trials rated as low in terms of risk of bias for all domains. However, they helpfully

provided recommendations for future trials to overcome this limitation (e.g. ITT designs,

CONSORT guidelines).

One of the main discussion points was that CBT was most effective at improving

daily functioning, and this led to speculations about what might have led to this. The effects

of exposure were discussed as a potential explanation. However, upon further examination of

the individual studies, not all “CBT” studies included exposure. The studies by Vollmer and

Blanchard (1998) and Labus et al (2013) involved no exposure at all; neither did the

mindfulness studies (Zernicke et al, 2013, Gaylord et al, 2011). As such, the authors’

speculations perhaps did not belong in this context.

The authors discussed the therapeutic potential of online interventions, despite saying

in the results section that they were unable to compare the effects on mental health of online

vs. in-person interventions, because only one trial provided mental health outcome data. They

5
 EIPCD Irene Lahde s2293048

did find a significant effect for daily functioning, but this was only from two online trials. As

such, they could have been more cautious in the discussion.

The authors were open about the inconsistency in reporting protocols and

demographics in RCTs. Again, they made sound recommendations to overcome this.

Although the article lacked a clear conclusion, the take-home message was that

psychotherapy significantly improved both mental health and daily functioning. However, the

authors admitted that only 18 trials provided daily functioning data. Given this, and the issue

with non-validated measures, the results relating to daily functioning should be interpreted

with caution. However, the authors recommended that future studies should consistently use

measures of daily functioning with good psychometric properties.

According to the authors, this was the first meta-analysis to investigate the effect of

psychotherapy on daily functioning. As such, this meta-analysis has value as a good starting

point for daily functioning literature, if the authors’ recommendations are taken forward.

References

Arch, J. J., & Ayers, C. R. (2013). Which treatment worked better for whom? Moderators of group

cognitive behavioral therapy versus adapted mindfulness-based stress reduction for anxiety

disorders. Behaviour Research and Therapy, 51(8), 434–442.

https://doi.org/10.1016/j.brat.2013.04.004

Ballou, S., & Keefer, L. (2017). The impact of irritable bowel syndrome on daily functioning:

Characterizing and understanding daily consequences of IBS. Neurogastroenterology &

Motility, 29(4), e12982. https://doi.org/10.1111/nmo.12982

6
 EIPCD Irene Lahde s2293048

Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2011). Introduction to meta-

analysis. John Wiley & Sons.

Cherkin, D. C., Sherman, K. J., Balderson, B. H., Cook, A. J., Anderson, M. L., Hawkes, R. J., …

Turner, J. A. (2016). Effect of mindfulness-based stress reduction vs cognitive behavioural

therapy or usual care on back pain and functional limitations in adults with chronic low back

pain: A randomized clinical trial. JAMA, 315(12), 1240.

https://doi.org/10.1001/jama.2016.2323

Donner, A., & Klar, N. (2004). Pitfalls of and controversies in cluster randomization trials.

American Journal of Public Health, 94(3), 416–422.

Egger, M., Zellweger-Zähner, T., Schneider, M., Junker, C., Lengeler, C., & Antes, G. (1997).

Language bias in randomised controlled trials published in English and German. Lancet

(London, England), 350(9074), 326–329. https://doi.org/10.1016/S0140-6736(97)02419-7

Gaylord, S. A., Palsson, O. S., Garland, E. L., Faurot, K. R., Coble, R. S., Mann, J. D., …

Whitehead, W. E. (2011). Mindfulness training reduces the severity of irritable bowel

syndrome in women: results of a randomized controlled trial. The American Journal of

Gastroenterology, 106(9), 1678–1688. https://doi.org/10.1038/ajg.2011.184

Guthrie, E., Creed, F., Dawson, D., & Tomenson, B. (1991). A controlled trial of psychological

treatment for the irritable bowel syndrome. Gastroenterology, 100(2), 450–457.

Higgins J. P. T., & Green, S. (editors). (2011). Cochrane Handbook for Systematic Reviews of

Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration. Available

from http://handbook.cochrane.org.

Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency

in meta-analyses. BMJ (Clinical Research Ed.), 327(7414), 557–560.

https://doi.org/10.1136/bmj.327.7414.557

7
 EIPCD Irene Lahde s2293048

Labus, J., Gupta, A., Gill, H. K., Posserud, I., Mayer, M., Raeen, H., … Mayer, E. A. (2013).

Randomised clinical trial: symptoms of the irritable bowel syndrome are improved by a

psycho-education group intervention. Alimentary Pharmacology & Therapeutics, 37(3), 304–

315. https://doi.org/10.1111/apt.12171

Laird, K. T., Tanner-Smith, E. E., Russell, A. C., Hollon, S. D., & Walker, L. S. (2017).

Comparative efficacy of psychological therapies for improving mental health and daily

functioning in irritable bowel syndrome: A systematic review and meta-analysis. Clinical

Psychology Review, 51, 142–152. https://doi.org/10.1016/j.cpr.2016.11.001

Moss-Morris, R., McAlpine, L., Didsbury, L. P., & Spence, M. J. (2010). A randomized controlled

trial of a cognitive behavioural therapy-based self-management intervention for irritable

bowel syndrome in primary care. Psychological Medicine, 40(01), 85.

https://doi.org/10.1017/S0033291709990195

Schröder, A., Rehfeld, E., Ørnbøl, E., Sharpe, M., Licht, R. W., & Fink, P. (2012). Cognitive–

behavioural group treatment for a range of functional somatic syndromes: Randomised trial.

British Journal of Psychiatry, 200(06), 499–507. https://doi.org/10.1192/bjp.bp.111.098681

Vollmer, A., & Blanchard, E. B. (1998). Controlled comparison of individual versus group

cognitive therapy for irritable bowel syndrome. Behavior Therapy, 29(1), 19–33.

https://doi.org/10.1016/S0005-7894(98)80016-6

Zernicke, K. A., Campbell, T. S., Blustein, P. K., Fung, T. S., Johnson, J. A., Bacon, S. L., &

Carlson, L. E. (2013). Mindfulness-based stress reduction for the treatment of irritable bowel

syndrome symptoms: A randomized wait-list controlled trial. International Journal of

Behavioral Medicine, 20(3), 385–396. https://doi.org/10.1007/s12529-012-9241-6