Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more
Standard view
Full view
of .
Look up keyword
Like this
0 of .
Results for:
No results containing your search query
P. 1
Journal club 5: observational studies

Journal club 5: observational studies

Ratings: (0)|Views: 31|Likes:
Reid, J. (Autumn 2011)
The fifth in our series to take the mystery out of critical appraisal looks at articles based on observational studies
Jennifer Reid’s series aims to help readers access the speech and language therapy literature, assess its credibility and decide how to act upon their findings. The content is based on the critical appraisal education format which has evolved in Fife and is delivered through a series of small group journal clubs. This article considers observational studies, how they differ from experimental ones, and the importance of recognising that evidence of an association is not evidence of causality. Three observational designs (cohort study, case-controlled study, and cross-sectional survey) are discussed. Two separate 10 question critical appraisal frameworks accompany this article: one for observational designs in general, and one for surveys that use questionnaires.
Reid, J. (Autumn 2011)
The fifth in our series to take the mystery out of critical appraisal looks at articles based on observational studies
Jennifer Reid’s series aims to help readers access the speech and language therapy literature, assess its credibility and decide how to act upon their findings. The content is based on the critical appraisal education format which has evolved in Fife and is delivered through a series of small group journal clubs. This article considers observational studies, how they differ from experimental ones, and the importance of recognising that evidence of an association is not evidence of causality. Three observational designs (cohort study, case-controlled study, and cross-sectional survey) are discussed. Two separate 10 question critical appraisal frameworks accompany this article: one for observational designs in general, and one for surveys that use questionnaires.

More info:

Published by: Speech & Language Therapy in Practice on Aug 05, 2013
Copyright:Attribution Non-commercial


Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less





 Journal club 5:
observational studies
Jennier Reid’sseries aims to help you access the speech and languagetherapy literature, assess its credibility and decide how to act on yourndings. Each instalment takes the mystery out o critically appraising adiferent type o journal article. Here, she looks at observational studies.
esearch is undamentally a quest orexplanations. Explanations go beyondsimple description in order to provide anaccount o causal relationships, or example,between events, human belies, behaviour,experiences and ill-health. It is really importantto keep the notion o causalityin mind as wetry to get our heads round the observationaldesigns used in health-related research. Causalreasoning, in a nutshell, requires us to:know what conditions preceded thephenomenon o interest,• assess which o these antecedentconditions are candidates as causalagents, and thenorganise this knowledge into a plausible,causal chain o events. There are a number o observational researchdesigns, and not all provide robust evidence o causality. It is not enough to demonstrate thatthere is anassociationbetween two actors.I you nd that children living in high atshave poorer health, this is not good evidencethat living in a at causes ill-health. The twoactors may be related, with economic orsocial circumstances perhaps much bettercandidates or an underlying cause.
Experimental or observational?
When an article talks about an intervention,how do I know i this is an experimental studyor an observational one?Group intervention studies, especially thoseusing randomisation to groups, are consideredsuperior to observational designs oranswering causal questions about healthcareinterventions, since there is better control o the efects o any unoreseen (conounding)actors. However, such anexperimentaldesign is not always practical or possible,especially where the participants’ commoncharacteristic cannot be manipulated (such ashaving a genetic condition) or would not beethical because o likely negative efects (orexample, not talking to your baby).Observationalstudies examine aspectso people’s past and/or present lie in orderto identiy relevant inormation throughobservation rather than experimental man-ipulation. They do not ofer a particularintervention and measure
its efect.However, inormation may be collected aboutintervention(s) the participants have received. The goal o observational studies is usuallyto identiy actors which
be causallyrelated, and thus may be incorporated intointerventions which then produce betteroutcomes in the uture.Sometimes people who have received aparticular intervention are ollowed up andtheir outcomes compared to the outcomes o others who did not receive the intervention.An observational study appraisal tool is likelyto be the one to choose or such a study. Theexception is i the intervention was oferedas a core part o the research design andparticipants were allocated or selected toreceive one intervention or another accordingto some preset criteria.
Observational designs
 There are a number o observational designsand some jargon to deal with.1. Cohort studyA cohort study ollows over time o a groupo people who have something in common.(The term cohort was also used to reer to agroup o Roman soldiers, which could be auseul aide-mémoire.) The group may havea common characteristic, such as wherethe participants were born, how they wereeducated or an aspect o their health or well-being. Alternatively they may have all beenexposed to a risk or challenging circumstanceo some kind, or have received a particularhealth intervention. The comparison group or cohort studiesmay be the general population rom which thecohort is drawn, or another cohort o personsthought to be similar except or the commoncharacteristic under investigation. Alternatively,subgroups within the cohort may be comparedwith each other. This is commonly the case inbirth cohortstudies, where all children bornin particular years in one geographical areaare studied. Results are analysed to detect acohort efect. This means nding out whethermembership o the cohort, and thereorehaving the common characteristic, appears tomake a diference to the outcome.In research designed to investigate riskso adverse health outcomes, a cohort isidentied
the appearance o thecondition(s) under investigation. For example,Conti-Ramsden & Botting (2007), in a studyo emotional health in adolescents, describetheir young people with specic languageimpairment as, “originally recruited at 7 yearso age as part o a wider study … The original
NHS Fie’s Dunermline cluster Journal Club. Author Jen Reid has her back to the camera.
cohort o 242 children represented a random50% sample o all children attending year 2(age 7) in language units across England.” Theemotional mental health o this cohort is thehealth outcome o interest. It is compared witha matched group o young people withouta history o specic language impairmentto explore whether there may be a greaterrisk o negative mental health outcomes oryoung people who have the condition.Cohort designs are particularly useul orstudying developmental changes across theliespan, such as to identiy the inuence o early circumstances, or the negative long-term efects o a condition, on lie outcomes.An example might be language and literacyoutcomes or children born very prematurely.However, they are expensive to do: outcomesmay take a long time to occur so you need toollow up the same group over a long periodo time, it is hard to prevent loss o participants(attrition) which is bad or the integrity o your results and, unless your cohort is verylarge indeed, it may be impossible to pick upenough people with a rare outcome to gainevidence or prognosis.2. Case-controlled studyIn a case-controlled (or case-control) study, onthe other hand, people who have the outcomeo interest (cases) are identied and matchedwith people who do not (controls). Forexample, a case-controlled design or Conti-Ramsden & Botting’s outcome o interest– emotional mental health in adolescence– might be to recruit participants with pooremotional mental health. They would theninvestigate their current language skills and / or their developmental language history incomparison with a matched group with goodemotional mental health. Case-controlledstudies may be the only practical design orresearching rare conditions or outcomes,but on their own they provide much weakerevidence o a causal relationship becausethere is much more risk o systematic biasafecting the results. You need to ensure everyparticipant is allocated correctly as a caseor not, as any misallocation can prooundlyinuence the results. The measures used todetermine who is a ‘case’ thereore need tobe pretty bullet-proo. This can be particularlytricky with complex human behaviour such ascommunication or emotional mental health.At a recent conerence I attended, speech andlanguage therapists debated whether theywould identiy the same children as language-delayed as the team studying a large Australianpreschool birth cohort. The study was using acut-of o 1.25 standard deviations below themean or their age on language testing. Weconcluded its ‘cases’ might include quite a ewo our ‘non-cases’.3. Cross-sectional survey The third main observational design is thecross-sectional survey. A representativesample o the population o interest (clients,practitioners, relatives) is interviewed,examined or otherwise studied to gaininormation on a question, such as, “Howmany children entering primary schoolhave poor vocabulary?” or, “What inuencesspeech and language therapy intervention oradults with autism and learning disability?“or, “What do care staf in residential homesknow about aphasia?” The data or cross-sectional studies are collected at a singlepoint in time. However, the study may includeretrospective inormation. An example wouldbe, in a survey o knowledge and skills ormaking inormation accessible or peoplewith learning disability, asking supportstaf whether they had ever received anyormal training on making inormationaccessible. Surveys can be relatively cheapand easy to do, but there are even morepotential challenges to the integrity o thedata, so it is not an appropriate design oranswering causal questions. I have prepared aseparate ramework tool or surveys that usequestionnaires, which is available at www.speechmag.com/Members/CASLT.For observational study results, there areways to evaluate how robust the evidence isor inerring causality. Remember, just becauseyou have established an association betweentwo actors, this does not allow you to assumea causal relationship. In terms o the numbers,acorrelation coecientsuch as Pearson’s r or
Figure 1 Criteria or Causation, rom Bradord Hill (1965)Temporal relationship:Cause always precedes the outcome.Strength:The stronger the association, the more likely it is that the relationship is causal.(NB Look at the signicance o those correlation coecients!)Consistency:The association is consistent across diferent studies.Dose-response relationship:An increasing amount o the proposed cause increases theoutcome’s severity or risk o occurrence.Sense:The causal explanation is theoretically plausible and compatible with currentknowledge.Alternate explanations:Other plausible explanations have been ruled out.Experiment:The outcome can be inuenced by an appropriate intervention.Specicity:A single putative cause produces a specic efect. This one is probably lessimportant or our purposes, especially given the multi-actorial nature o most o thebehaviours speech and language therapists are dealing with.
Spearman’s rho only indicates the presenceor absence and direction o any associationbetween the variables being measured.Hill’s (1965) criteria or causation (gure 1)were originally designed or epidemiologicalstudies but have been widely quoted andso may pop up in authors’ discussion o theresults o their observational studies.Here is an appropriately cautious conclusionabout causality rom the Conti-Ramsden &Botting (2007) study: “Our data show a clearincreased risk or this population as theynear adulthood compared to peers, evenwhen concurrent language and cognition areaccounted or. This nding replicates otherstudies that have shown raised prevalenceo psychiatric diculties in those withcommunication impairments … or increasedlanguage impairment in children reerredpsychiatrically … However, the associationhas oten been assumed to be causal in thateither long-term language impairment maylead to (or exacerbate) wider dicultiesor psychiatric impairment may constraincommunication skill. Nonetheless, it needsto be noted that the majority o adolescentswith SLI in our study did not appear to suferrom emotional problems” (p. 522).
 The reporting o observational studies in peer-reviewed journals has been inuenced bythe STROBE statement (von Elm
et al.
, 2008).Like the Bradord Hill criteria (gure 1), thiswas originally devised to improve reportingo epidemiological research but it has beenextended to other areas. Although designedor authors, it may provide some guidanceor readers too. Observational studiesare perhaps less common in speech andlanguage therapy literature, so I have oundit helpul to have a tool that encapsulates allthe main observational designs rather thantrying to match separate tools to cohort,case-controlled and cross-sectional studies. Ideveloped the ollowing appraisal ramework or speech and language therapists romrelevant CASP tools (PHRU, 2006) and theSTROBE statement (combined checklist).
Critical appraisal forspeech and languagetherapists (CASLT)
 Download the observationalstudy and surveyquestionnaire frameworksfrom www.speechmag.com/Members/CASLTfor your own use or withcolleagues in a journal club. 
Question 1:Did the study address a clearly ocused issue?Which population was studied? Which risk actors or outcomes were investigated? Didthe study try to detect a benecial or harmulefect? Is the underlying issue one o caus-ation? Try ormulating the reviewers’ statedaims into a research question i they have notdone so explicitly in the article. Is this questionimportant or your clinical practice?Question 2:Was the choice o design appropriate?Is an observa-tional design anappropriate wayo answering theresearch questionunder the circum-stances? Remem-ber that a groupintervention studyis a more poweruldesign or demon-strating causality. Try to work out whether thisis a cohort, case-controlled or cross-sectionalstudy. For a cohort study, the participantsshould have been recruited beore the out-come o interest has occurred, and the cohortshould have something in common (thoughthis can be a very general characteristic, suchas being born in Scotland in 2005, or a morespecic one like being a sibling o a child withautism spectrum disorder). For a case-controlledstudy, ‘cases’ and ‘controls’ are identied at theoutset o the study, criteria or ‘caseness’ arecrucial or quality control, and the outcomeo interest should be rare or harmul. Cross-sectional surveys are probably the easiest tospot, since we meet them regularly in eve-ryday lie. For surveys, sampling methodswhich ensure that the participants representadequately the population o interest are veryimportant or quality control.Question 3:Were participants recruited in an acceptable way?In general, you are looking or selection biaswhich might compromise the extent to whichthe ndings can be generalised (externalvalidity). Were participants representativeo a clearly dened and clinically relevantpopulation? Appraise the eligibility (inclusionand exclusion) criteria, the sources andmethods o selection o participants andhow cohorts were ollowed up. The selectionmethod should be systematic – explicit,reliable and replicable - especially or a case-controlled study, where it is crucial that there isno misallocation o cases.Scrutinise also the way that controls havebeen selected. Are they matched, population-based or randomly selected, and is therationale justied? I controls were matched,were the matching criteria appropriate?Authors should also provide inormation toallow you to assess whether those who wereinvited to participate but declined could bediferent in any important way rom the studyparticipants. Potential controls are perhapsmore likely to decline or ignore invitationsto participate, so this may be even moreimportant or this group.How many participants were there, wasthere a rationale or this and were thenumbers sucient to support generalisationo the ndings?I the study is asking, “How many peoplehave…” you need to think whether thesampling is o newly identied (incidence)or o cases across the whole population(prevalence) (see gure 2), and which wouldprovide a more appropriate answer to theresearch question.Question 4:Were phenomena measured accuratelyenough to minimise bias?You should be given enough inormation toassess how well all the phenomena involvedhave been assessed or otherwise measured,both actors (cohort characteristics or casenesscriteria) and outcomes. Are denitions clearenough? Were measurements subjective orobjective, and do they measure what they aresupposed to measure? Externally validatedmeasures, like ormal tests, need less supportinginormation than measures developed or thepurposes o the study.Some outcomes may take a long time to occur,so was the timerame o the study long enoughto assess this accurately or all participants?Moreover, the participants who are lost toollow-up may have diferent outcomes romthose who were available, so attrition rates needto be given and their potential impact discussed.A owchart o recruitment and ollow-upschedule, indicating attrition numbers, can bereally helpul or long-term studies.As in intervention trials, the study methodshould minimise the possibility o perormancebias by employing similar measurementmethods or both cases and controls, and byblinding those undertaking the assessmentsto participants’ status wherever easible.Question 5:Has there been adequate attention toconounding?Conounding is the inuence o unoreseenactors. Check which actors have beenconsidered and list any you think might beimportant that the authors seem to haveoverlooked. How, i at all, have the researcherstaken account o the conounding actors inthe design and/or analysis? Have a look in thedata analysis section or evidence they haveused statistical techniques such as modelling,regression or sensitivity analysis to makeadjustments or conounding actors. Here issome relevant wording rom the Conti-Ramsden& Botting (2007) article: “… all the analyses abovecomparing those with SLI and those with NLD[no language disorder] remained unchangedater controlling or gender…”Question 6:What are the results o this study?As or inter-vention studies,it helps to tryto sum up thebottom-lineresult o thestudy in onesentence – thisalso helps to
Figure 2 Incidence or Prevalence?iNCidence is the number o NewCases in a given time spanPrevalence is theProportion o cases in the Population

You're Reading a Free Preview

/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->