You are on page 1of 15

Ear, nose, and throat disorders

..................................................

Sinusitis (acute rhinosinusitis)


Search date October 2013
Kim Ah-See

ABSTRACT
INTRODUCTION: Acute rhinosinusitis is defined pathologically by transient inflammation of the mucosal lining of the paranasal sinuses
lasting less than 4 weeks. Clinically, it is characterised by nasal congestion, rhinorrhoea, facial pain, hyposmia, sneezing, and, if more severe,
additional malaise and fever. It affects 1% to 5% of the adult population each year in Europe. METHODS AND OUTCOMES: We conducted
a systematic review and aimed to answer the following clinical question: What are the effects of treatments in people with clinically diagnosed
acute rhinosinusitis? We searched: Medline, Embase, The Cochrane Library, and other important databases up to October 2013 (BMJ
Clinical Evidence reviews are updated periodically; please check our website for the most up-to-date version of this review). We included
harms alerts from relevant organisations such as the US Food and Drug Administration (FDA) and the UK Medicines and Healthcare products
Regulatory Agency (MHRA). RESULTS: We found 12 studies that met our inclusion criteria. We performed a GRADE evaluation of the
quality of evidence for interventions. CONCLUSIONS: In this systematic review we present information relating to the effectiveness and
safety of the following interventions: antibiotics (amoxicillin, amoxicillin-clavulanic acid [co-amoxiclav], doxycycline, cephalosporins, macrolides;
long-course regimens), corticosteroids (intranasal), decongestants (xylometazoline, phenylephrine, pseudoephedrine), and saline nasal
washes.

QUESTIONS
What are the effects of treatments in people with clinically diagnosed acute rhinosinusitis?. . . . . . . . . . . . . . . . 3

INTERVENTIONS
CLINICALLY DIAGNOSED ACUTE SINUSITIS macrolides only; limited evidence compared with placebo
in people with mild-to-moderate acute rhinosinusitis, but
Likely to be beneficial may be associated with increased adverse effects) . .
Corticosteroids (intranasal) . . . . . . . . . . . . . . . . . . . 3 6
Decongestants (xylometazoline, phenylephrine, pseu-
Unknown effectiveness doephedrine) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Selected antibiotics (amoxicillin, amoxicillin-clavulanic Saline nasal washes . . . . . . . . . . . . . . . . . . . . . . . 13
acid [co-amoxiclav], doxycycline, cephalosporins,

Key points

• Acute rhinosinusitis is defined pathologically by transient inflammation of the mucosal lining of the paranasal sinuses
lasting less than 4 weeks.
It affects 1% to 5% of the adult population each year in Europe.
Characteristic symptoms include nasal congestion, rhinorrhoea, facial pain, hyposmia, sneezing, and, if more
severe, additional malaise and fever.
The diagnosis is usually made clinically without radiological or bacteriological investigation. Clinically diagnosed
acute rhinosinusitis is less likely to be caused by bacterial infection than acute rhinosinusitis confirmed by radio-
logical or bacteriological investigation.
• This review examines evidence from RCTs and systematic reviews of RCTs in adults with clinically diagnosed
acute rhinosinusitis only. We excluded studies in people with acute rhinosinusitis confirmed by radiological or
bacterial investigation.
• In clinically diagnosed acute rhinosinusitis, intranasal corticosteroids may reduce symptoms compared with
placebo.
• We found little RCT evidence to support the use of amoxicillin, co-amoxiclav, or doxycycline in people with clinically
diagnosed acute rhinosinusitis.
• We found no RCTs comparing cephalosporins or macrolides with placebo in people with clinically diagnosed acute
rhinosinusitis.
• The balance between little evidence of benefit from antibiotics versus increased adverse effects should be discussed
when considering prescribing antibiotics for people with mild-to-moderate clinically diagnosed acute rhinosinusitis.
Incidence of adverse effects is higher in people prescribed antibiotics compared with placebo.
The clinical response from cefotiam (a cephalosporin) appears to be the same after a 5-day course versus a 10-
day course. However, this single RCT did not compare outcome with placebo.
We found no RCTs on the effects of antibiotics in people with severe clinically diagnosed acute rhinosinusitis.
• We found no good-quality RCTs examining the effectiveness of decongestants or saline nasal washes in acute
rhinosinusitis diagnosed clinically.

© BMJ Publishing Group Ltd 2015. All rights reserved. ..................... 1 ..................... Clinical Evidence 2015;04:511
Ear, nose, and throat disorders
Sinusitis (acute rhinosinusitis)
Clinical context

GENERAL BACKGROUND
Acute rhinosinusitis is a commonly clinically diagnosed condition in primary care. It affects 1% to 5% of the adult
population each year in Europe.

FOCUS OF THE REVIEW


Antibiotics are frequently prescribed for this condition. This review focuses on the evidence for the use of topical
corticosteroids and antibiotics in this condition.

COMMENTS ON EVIDENCE
The evidence presented comes from RCTs and systematic reviews of RCTs. However, the overall quality of the ev-
idence from these studies is low.

SEARCH AND APPRAISAL SUMMARY


The update literature search for this review was carried out from the date of the last search, June 2011, to October
2013. For more information on the electronic databases searched and criteria applied during assessment of studies
for potential relevance to the review, please see the Methods section. Searching of electronic databases retrieved
125 studies. Appraisal of titles and abstracts led to the exclusion of 113 studies and the further review of 12 full
publications. Of the 12 full articles evaluated, three systematic reviews and one RCT were added at this update.

ADDITIONAL INFORMATION
No studies included the treatment of severe acute rhinosinusitis; the conclusions are limited to mild-to-moderate
clinically diagnosed acute rhinosinusitis. The incidence of adverse effects appears to be higher in people treated
with antibiotics compared to placebo.

DEFINITION Acute rhinosinusitis is defined pathologically by transient inflammation of the mucosal lining of the
paranasal sinuses lasting less than 4 weeks. Clinically, it is characterised by nasal congestion,
rhinorrhoea, facial pain, hyposmia, sneezing, and, if more severe, by additional malaise and fever.
The diagnosis is usually made clinically (on the basis of history and examination, but without radi-
ological or bacteriological investigation). Clinically diagnosed acute rhinosinusitis is less likely to
be caused by bacterial infection than acute rhinosinusitis confirmed by radiological or bacteriological
[1]
investigation. In this review, we have excluded studies in people with acute rhinosinusitis con-
firmed by bacteriological or radiological investigation, in children (younger than 16 years of age),
in people with symptoms for longer than 4 weeks (chronic sinusitis) or recurrent sinusitis (may in-
dicate a structural problem in the nasal cavity), and in people with symptoms after facial trauma.

INCIDENCE/ Each year in Europe, 1% to 5% of adults are diagnosed with acute rhinosinusitis by their general
[2]
PREVALENCE practitioners. Extrapolated to the British population, this is estimated to cause 6 million restricted
[3] [4]
working days per year. Most people with acute rhinosinusitis are assessed and treated in a
primary-care setting. The prevalence varies according to whether diagnosis is made on clinical
grounds or on the basis of radiological or bacteriological investigation.

AETIOLOGY/ One systematic review (search date 1998) reported that about 50% of people with a clinical diag-
[1]
RISK FACTORS nosis of acute rhinosinusitis have bacterial sinus infection. The usual pathogens in acute bacte-
rial rhinosinusitis are Streptococcus pneumoniae and Haemophilus influenzae, with occasional in-
[5]
fection with Moraxella catarrhalis. Preceding viral upper respiratory-tract infection is often the
[6]
trigger for acute bacterial rhinosinusitis, with about 0.5% of common colds becoming complicated
[7]
by the development of acute rhinosinusitis.

PROGNOSIS One meta-analysis of RCTs found that up to two-thirds of people with acute rhinosinusitis had
[8]
spontaneous resolution of symptoms without active treatment. One non-systematic review re-
ported that people with acute rhinosinusitis are at risk of chronic rhinosinusitis and irreversible
[9]
damage to the normal mucociliary mucosal surface. One further non-systematic review reported
rare life-threatening complications, such as orbital cellulitis and meningitis, after acute rhinosinusitis.
[10]
However, we found no reliable data to measure these risks.

AIMS OF To relieve symptoms as quickly as possible, with minimal adverse effects.


INTERVENTION

© BMJ Publishing Group Ltd 2015. All rights reserved. ........................................................... 2


Ear, nose, and throat disorders
Sinusitis (acute rhinosinusitis)
OUTCOMES Improvement in sinusitis includes symptom improvement (symptom scores), cure rates (time to
self-reported symptom resolution and time to clinical resolution [defined by examiner]), and time
to return to normal activities; quality of life; and adverse effects. In the identified studies, clinical
improvement and clinical cure were often used as outcome measures. 'Clinical improvement' was
defined as improvement in clinical state as rated by the assessor or by the participant. 'Clinical
cure' was defined as resolution of symptoms as rated by assessor or participant.

METHODS BMJ Clinical Evidence search and appraisal October 2013. The following databases were used to
identify studies for this systematic review: Medline 1966 to October 2013, Embase 1980 to October
2013, and The Cochrane Database of Systematic Reviews 2013, Issue 10 (1966 to date of issue).
Additional searches were carried out in the Database of Abstracts of Reviews of Effects (DARE)
and the Health Technology Assessment (HTA) Database. We also searched for retractions of
studies included in the review. Titles and abstracts identified by the initial search run by an informa-
tion specialist were first assessed against predefined criteria by an evidence scanner. Full texts
for potentially relevant studies were then assessed against predefined criteria by an evidence an-
alyst. Studies selected for inclusion were discussed with an expert contributor. All data relevant to
the review were then extracted by an evidence analyst. Study design criteria for inclusion in this
review were: published systematic reviews of RCTs and RCTs in the English language, at least
single-blinded, and containing at least 20 individuals of whom at least 80% were followed up. The
minimum length of follow-up required to include studies was 1 week. We excluded all studies that
included people with clinical symptoms that were also confirmed by bacteriological or radiological
investigation, as well as studies described as 'open', 'open label', or not blinded unless blinding
was impossible. We included systematic reviews of RCTs and RCTs where harms of an included
intervention were assessed, applying the same study design criteria for inclusion as we did for
benefits. In addition, we use a regular surveillance protocol to capture harms alerts from organisa-
tions such as the FDA and the MHRA, which are added to the reviews as required. To aid readabil-
ity of the numerical data in our reviews, we round many percentages to the nearest whole number.
Readers should be aware of this when relating percentages to summary statistics such as relative
risks (RRs) and odds ratios (ORs). We have performed a GRADE evaluation of the quality of evi-
dence for interventions included in this review (see table, p 15 ). The categorisation of the quality
of the evidence (high, moderate, low, or very low) reflects the quality of evidence available for our
chosen outcomes in our defined populations of interest. These categorisations are not necessarily
a reflection of the overall methodological quality of any individual study, because the Clinical Evi-
dence population and outcome of choice may represent only a small subset of the total outcomes
reported, and population included, in any individual trial. For further details of how we perform the
GRADE evaluation and the scoring system we use, please see our website (www.clinicalevi-
dence.com).

QUESTION What are the effects of treatments in people with clinically diagnosed acute rhinosinusitis?

OPTION CORTICOSTEROIDS (INTRANASAL) VERSUS PLACEBO. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

• For GRADE evaluation of interventions for Sinusitis (acute rhinosinusitis), see table, p 15 .
• In clinically diagnosed acute rhinosinusitis, intranasal corticosteroid spray may reduce symptoms at 14 to 15
days compared with placebo.

Benefits and harms


Corticosteroids (intranasal) versus placebo:
[11] [12] [13]
We found two systematic reviews (both with the search dates of 2011) and one subsequent RCT. The
first review identified four RCTs (1 RCT in people with clinically diagnosed rhinosinusitis and 3 RCTs in people with
[11]
radiologically or biologically confirmed rhinosinusitis). The review did not perform separate analyses in people
with clinically diagnosed or laboratory-confirmed rhinosinusitis; therefore, we report the results of only the RCT in
[14]
people with clinically confirmed rhinosinusitis. Assessment of symptom improvement in the review differed from
[12]
that in the RCT, and so we report both here. The second review identified six RCTs, one of which met the BMJ
[14]
Clinical Evidence inclusion criteria, and was also included in the first systematic review.

-
Improvement in sinusitis
Corticosteroids (intranasal) compared with placebo Intranasal corticosteroids (mometasone, fluticasone) may be
more effective than placebo at 14 to 15 days at improving symptom scores in people with clinically diagnosed acute
rhinosinusitis (very low-quality evidence).

© BMJ Publishing Group Ltd 2015. All rights reserved. ........................................................... 3


Ear, nose, and throat disorders
Sinusitis (acute rhinosinusitis)
Ref Results and statistical Effect
(type) Population Outcome, Interventions analysis size Favours
Symptom improvement
[11]
981 people with Resolution or improvement in RR 1.04 for mometasone furoate
clinically diagnosed symptoms , 15 days (once or twice-daily) v placebo
Systematic
acute rhinosinusitis
review 442/478 (93%) with mometasone 95% CI 0.99 to 1.09
in a primary-care
furoate nasal spray once or twice
setting
daily
[12]
In review
225/252 (89%) with placebo
Data from 1 RCT Not significant
Treatments were given for 15
[14]
4-armed trial; days
the remaining arm
Absolute numbers taken from re-
evaluated amoxi-
view, which combined data for
cillin
mometasone furoate once-daily
and mometasone furoate twice-
daily

[14]
981 people with Mean reduction of major Mean difference: 0.81
clinically diagnosed symptom score , 15 days
RCT P <0.001
acute rhinosinusitis
with mometasone furoate nasal
4-armed in a primary-care
spray (twice-daily)
trial setting
[11] [12] with placebo
In review
Absolute results not reported mometasone
Treatments were given for 15 furoate nasal spray
days; see Further information on
studies for description of major
symptom score
The remaining arms evaluated
mometasone furoate nasal spray
once-daily and amoxicillin

[13]
737 people with Reduction of major symptom Mean difference –0.4
uncomplicated score, least square mean
RCT 95% CI –0.67 to –0.10
acute rhinosinusitis change , 14 days
3-armed P = 0.008
–3.4 with fluticasone furoate
trial
nasal spray (once daily)
fluticasone furoate
–3.0 with placebo
nasal spray
The remaining arm evaluated
fluticasone furoate (twice-daily)
n = 240 fluticasone furoate (once
daily), n = 252 fluticasone furoate
(twice-daily), n = 245 placebo

[13]
737 people with Reduction of major symptom Mean difference –0.4
uncomplicated score, least square mean
RCT 95% CI –0.64 to –0.07
acute rhinosinusitis change , 14 days
3-armed P = 0.008
–3.3 with fluticasone furoate
trial
nasal spray (twice-daily)
fluticasone furoate
–3.0 with placebo
nasal spray
The remaining arm evaluated
fluticasone furoate (once daily)
n = 240 fluticasone furoate (once
daily), n = 252 fluticasone furoate
(twice daily), n = 245 placebo

-
Quality of life
Corticosteroids (intranasal) compared with placebo Intranasal fluticasone may be no more effective than placebo at
improving quality of life (measured by SNOT-20 score) at 14 days in people with clinically diagnosed acute rhinosi-
nusitis (low-quality evidence).

© BMJ Publishing Group Ltd 2015. All rights reserved. ........................................................... 4


Ear, nose, and throat disorders
Sinusitis (acute rhinosinusitis)
Ref Results and statistical Effect
(type) Population Outcome, Interventions analysis size Favours
Quality of life
[13]
737 people with Adjusted mean change in Mean difference –0.11
uncomplicated quality of life, least square
RCT 95% CI –0.26 to +0.04
acute rhinosinusitis mean change , 14 days
3-armed P = 0.142
–1.57 with fluticasone furoate
trial
nasal spray (once-daily) Analysis performed using analy-
sis of covariance (ANCOVA) with
–1.46 with placebo
baseline value, country, and aller-
The remaining arm evaluated gic rhinitis status as covariates
fluticasone furoate twice-daily Not significant
n = 240 fluticasone furoate (once
daily), n = 252 fluticasone furoate
(twice-daily), n = 245 placebo
Quality of life assessed using
SinoNasal Outcome Test (SNOT-
20); see Further information on
studies for description of SNOT-
20

[13]
737 people with Adjusted mean change in Mean difference: –0.14
uncomplicated quality of life, least square
RCT 95% CI –0.29 to 0.00
acute rhinosinusitis mean change , 14 days
3-armed P = 0.058
–1.60 with fluticasone furoate
trial
nasal spray (twice-daily) Analysis performed using analy-
sis of covariance (ANCOVA) with
–1.46 with placebo
baseline value, country, and aller-
The remaining arm evaluated gic rhinitis status as covariates
fluticasone furoate once-daily Not significant
n = 240 fluticasone furoate (once-
daily), n = 252 fluticasone furoate
(twice-daily), n = 245 placebo
Quality of life assessed using
SinoNasal Outcome Test (SNOT-
20); see Further information on
studies for description of SNOT-
20

-
Adverse effects

-
Ref Results and statistical Effect
(type) Population Outcome, Interventions analysis size Favours
Adverse effects
[14]
981 people with Treatment-related adverse ef- Significance not reported
clinically diagnosed fects , 15 days
RCT
acute rhinosinusitis
36% with mometasone furoate
4-armed in a primary-care
nasal spray (twice-daily)
trial setting
[11] 35% with mometasone furoate
In review
nasal spray (once-daily)
38% with placebo
Absolute numbers not reported
Treatments were given for 15
days
The most common adverse ef-
fects were headache and epis-
taxis
The remaining arm evaluated
amoxicillin

© BMJ Publishing Group Ltd 2015. All rights reserved. ........................................................... 5


Ear, nose, and throat disorders
Sinusitis (acute rhinosinusitis)
Ref Results and statistical Effect
(type) Population Outcome, Interventions analysis size Favours
[13]
737 people with Adverse effects during treat- Reported as similar across all
uncomplicated ment treatment groups
RCT
acute rhinosinusitis
41/240 (17%) with fluticasone P value not reported
3-armed
furoate nasal spray (once-daily)
trial
46/252 (18%) with fluticasone
furoate nasal spray (twice-daily)
41/245 (17%) with placebo
The most common adverse ef-
fects were headache, bacterial
sinusitis, and epistaxis
There were no deaths; 1 serious
adverse effect was reported,
which was not drug-related; the
RCT did not report which arm this
was in

-
-
-
Further information on studies
[14]
The RCT included people aged 12 years and older; 6% of participants were less than 16 years old. Major
symptom scores were based on the sum of individual scores (0 = none, 1 = mild, 2 = moderate, 3 = severe) for
symptoms that included rhinorrhoea, postnasal drip, nasal congestion, sinus headache, and facial pain. Follow-
up at 14 days after treatment suggested no greater risk of recurrence or exacerbation in the mometasone furoate
group compared with placebo (significance not assessed).
[13]
The RCT (n = 737) included people aged 12 years and older, with a mean age of 39 years. People with fulminant
bacterial rhinosinusitis clinically suggested by symptoms, including temperature higher than 38°C and persistent
severe facial/tooth pain, were excluded. They also excluded people with chronic or recurrent rhinosinusitis,
symptomatic allergic rhinitis, or allergic sensitisation to seasonal allergens likely to be present during the study
(determined by skin prick test or in vitro blood test). The effect of treatment on disease-specific quality of life
was measured using the SinoNasal Outcome Test-20 (SNOT-20); a questionnaire consisting of 20 individual
items, each rated using a 0 to 5 scale (0 = none, 1 = very mild, 2 = mild, 3 = moderate, 4 = severe, 5 = as bad
as it can be). Two of the nine authors were employees of the pharmaceutical company that had funded the trial.

-
-
Comment: Clinical guide:
In clinically diagnosed acute rhinosinusitis, 14 to 15 days of topical corticosteroid nasal spray may
confer a treatment benefit in terms of symptom reduction when compared with placebo.

OPTION ANTIBIOTICS (AMOXICILLIN, AMOXICILLIN-CLAVULANIC ACID [CO-AMOXICLAV], DOXY-


CYCLINE, CEPHALOSPORINS, MACROLIDES). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

• For GRADE evaluation of interventions for Sinusitis (acute rhinosinusitis), see table, p 15 .
• We found no RCTs on the effects of cephalosporins or macrolides compared with placebo in clinically diagnosed
acute rhinosinusitis.
• For clinically diagnosed mild to moderate acute rhinosinusitis there is little evidence to support the use of the
commonly used antibiotics, including amoxicillin and doxycycline.
• The incidence of adverse effects is higher in people prescribed antibiotics compared with placebo.
• The balance between little evidence of benefit from antibiotics versus increased adverse effects should be dis-
cussed when considering prescribing antibiotics for mild to moderate acute rhinosinusitis.
• The clinical response from cefotiam (a cephalosporin) appears to be the same after a 5-day course versus a 10-
day course. However, this single RCT did not compare outcome with placebo.

© BMJ Publishing Group Ltd 2015. All rights reserved. ........................................................... 6


Ear, nose, and throat disorders
Sinusitis (acute rhinosinusitis)
Benefits and harms
Amoxicillin versus placebo:
We found one systematic review, which identified six RCTs comparing amoxicillin with placebo in people with clini-
[15]
cally diagnosed acute rhinosinusitis (search date 2012; 1598 participants). One of the RCTs included in the
systematic review did not meet our inclusion criteria because it was designed to compare combinations of interventions.
The systematic review pooled data for all antibiotics (see Further information on studies); therefore, for multiple-
armed RCTs that included more than one antibiotic, we have reported directly from the RCT for head-to-head com-
parisons against placebo.

-
Improvement in sinusitis
Amoxicillin compared with placebo Amoxicillin may be no more effective than placebo at increasing treatment success
rates in people with clinically diagnosed acute rhinosinusitis (low-quality evidence).

Ref Results and statistical Effect


(type) Population Outcome, Interventions analysis size Favours
Cure rates
[15]
People with clinical- Cure (resolution or improve- OR 1.45
ly diagnosed acute ment of major symptoms,
Systematic 95% CI 0.95 to 2.21
rhinosinusitis in a evaluated by the participant
review
primary-care set- alone or the participant and the
Not significant
ting investigator) , at 10 days
Data from 1 RCT 73/189 (39%) with amoxicillin
59/195 (30%) with placebo

[15]
People with clinical- Cure (resolution or improve- OR 1.87
ly diagnosed acute ment of major symptoms,
Systematic 95% CI 0.89 to 3.90
rhinosinusitis evaluated by the participant
review
alone or the participant and the
Data from 1 RCT
investigator during telephone Not significant
interviews) , at 2 weeks
32/56 (57%) with amoxicillin
25/60 (42%) with placebo

[15]
People with clinical- Participants 'significantly im- OR 2.30
ly diagnosed acute proved' , at 7 days
Systematic 95% CI 1.17 to 4.52
rhinosinusitis in a
review 60/81 (74%) with amoxicillin amoxicillin
primary-care set-
ting 41/74 (55%) with placebo
Data from 1 RCT

[15]
People with clinical- Participants 'significantly im- OR 0.89
ly diagnosed acute proved' , at 10 days
Systematic 95% CI 0.41 to 1.93
rhinosinusitis in a
review 63/81 (77%) with amoxicillin Not significant
primary-care set-
ting 59/74 (80%) with placebo
Data from 1 RCT

[17]
150 people with Recovery rates (assessed by Significance not reported
clinically diagnosed telephone) , 2 weeks
RCT
acute maxillary rhi-
18/23 (78%) with amoxicillin
4-armed nosinusitis in a pri-
trial mary-care setting 39/59 (66%) with placebo
[15]
In review The remaining arms evaluated
doxycycline and penicillin

-
Quality of life
Amoxicillin compared with placebo We don’t know whether amoxicillin improves quality of life (measured by SNOT-
16 score) at up to 10 days compared with placebo in people with clinically diagnosed acute rhinosinusitis, as there
were limited data from one RCT and results varied over time (low-quality evidence).

© BMJ Publishing Group Ltd 2015. All rights reserved. ........................................................... 7


Ear, nose, and throat disorders
Sinusitis (acute rhinosinusitis)
Ref Results and statistical Effect
(type) Population Outcome, Interventions analysis size Favours
Quality of life
[15]
People with clinical- Treatment success (defined as OR 2.30
ly diagnosed acute significant symptom improve-
Systematic 95% CI 1.17 to 4.52
rhinosinusitis in a ment on SNOT-16) , 7 days
review amoxicillin
primary-care set- P = 0.02
60/81 (74%) with amoxicillin
ting
41/74 (55%) with placebo
Data from 1 RCT

[15]
People with clinical- Treatment success (defined as OR 0.89
ly diagnosed acute significant symptom improve-
Systematic 95% CI 0.41 to 1.93
rhinosinusitis in a ment on SNOT-16) , 10 days
review Not significant
primary-care set- P = 0.71
63/81 (78%) with amoxicillin
ting
59/74 (80%) with placebo
Data from 1 RCT

-
Adverse effects

-
Ref Results and statistical Effect
(type) Population Outcome, Interventions analysis size Favours
Adverse effects
[15]
People with clinical- Diarrhoea Peto OR 1.74
ly diagnosed acute
Systematic 55/189 (29%) with amoxicillin 95% CI 1.09 to 2.78
rhinosinusitis in a
review placebo
primary-care set- 37/195 (19%) with placebo
ting
Data from 1 RCT

[15]
116 people with Diarrhoea , 2 weeks OR 3.73
clinically diagnosed
Systematic 4/56 (7%) with amoxicillin 95% CI 0.63 to 22.24 Not significant
acute rhinosinusitis
review
1/60 (2%) with placebo
Data from 1 RCT

-
-
Doxycycline versus placebo:
We found one systematic review (search date 2012) that included two RCTs (342 people) comparing doxycycline
[15]
with placebo in people with clinically diagnosed acute rhinosinusitis. The systematic review pooled data for all
antibiotics (see Further information on studies); therefore, for multiple-armed RCTs that included more than one
antibiotic we have reported directly from the RCT for head-to-head comparisons against placebo.

-
Improvement in sinusitis
Doxycycline compared with placebo Doxycycline may be no more effective than placebo at increasing cure rates at
10 days, at improving the time to resolution of facial pain, or the time to return to normal activities in people with
clinically diagnosed acute rhinosinusitis (very low-quality evidence).

Ref Results and statistical Effect


(type) Population Outcome, Interventions analysis size Favours
Cure rates
[15]
192 people with Cure rate , 10 days OR 0.95
clinically diagnosed
Systematic 56/94 (60%) with doxycycline 95% CI 0.53 to 1.70
acute rhinosinusitis
review
in a primary-care 56/92 (61%) with placebo
setting
Both groups were also given xy- Not significant
Data from 1 RCT lometazoline nose drops and
steam inhalation
Overall, 60% of people in the
RCT were cured at 10 days

© BMJ Publishing Group Ltd 2015. All rights reserved. ........................................................... 8


Ear, nose, and throat disorders
Sinusitis (acute rhinosinusitis)
Ref Results and statistical Effect
(type) Population Outcome, Interventions analysis size Favours
[17]
150 people with Recovery rates (assessed by Significance not reported
clinically diagnosed telephone) , 2 weeks
RCT
acute maxillary rhi-
26/33 (79%) with doxycycline
4-armed nosinusitis in a pri-
trial mary-care setting 39/59 (66%) with placebo
[15]
In review The remaining arms evaluated
amoxicillin and penicillin

[20]
192 people with Cure rate , 42 days Significance not reported
clinically diagnosed
RCT with doxycycline
acute rhinosinusitis
in a primary-care with placebo
setting
Absolute results not reported
[15]
In review
Both groups were also given xy-
lometazoline nose drops and
steam inhalation
Overall, 90% of people in the trial
were cured at 42 days
The RCT reported that there was
no difference between groups in
cure rate

Symptom improvement
[20]
192 people with Median time to resolution of HR 1.17
clinically diagnosed facial pain
RCT 95% CI 0.87 to 1.57
acute rhinosinusitis
4 days with doxycycline
in a primary-care
setting 5 days with placebo Not significant
[15]
In review Both groups were also given xy-
lometazoline nose drops and
steam inhalation

Time to return to normal activities


[20]
192 people with Median time to return to normal HR 1.31
clinically diagnosed activities
RCT 95% CI 0.96 to 1.78
acute rhinosinusitis
5 days with doxycycline Not significant
in a primary-care
setting 6 days with placebo
[15]
In review

-
Adverse effects

-
Ref Results and statistical Effect
(type) Population Outcome, Interventions analysis size Favours
Adverse effects
[15]
192 people with Adverse effects OR 9.94
clinically diagnosed
Systematic 17/94 (18%) with doxycycline 95% CI 2.22 to 44.37
acute rhinosinusitis
review
in a primary-care 2/92 (2%) with placebo placebo
setting
Adverse effects included nausea,
Data from 1 RCT vomiting, abdominal pain, diar-
rhoea, and rash

-
[17]
No data from the following reference on this outcome.

-
-

© BMJ Publishing Group Ltd 2015. All rights reserved. ........................................................... 9


Ear, nose, and throat disorders
Sinusitis (acute rhinosinusitis)
Amoxicillin-clavulanic acid (co-amoxiclav) versus placebo:
[15] [21]
We found one systematic review (search date 2012) that included one RCT (252 participants).

-
Improvement in sinusitis
Amoxicillin-clavulanic acid (co-amoxiclav) compared with placebo Amoxicillin-clavulanic acid (co-amoxiclav) may be
no more effective than placebo at reducing the time to return to normal activities or at increasing the proportion of
people with cure at 1 week in people with clinically diagnosed acute rhinosinusitis (low-quality evidence).

Ref Results and statistical Effect


(type) Population Outcome, Interventions analysis size Favours
Time to return to normal activities
[21]
252 people with Time to return to normal activi- HR (adjusted for specified vari-
clinically diagnosed ties ables) 0.99
RCT
acute rhinosinusitis
with amoxicillin-clavulanic acid 95% CI 0.68 to 1.45
recruited from gen- Not significant
(co-amoxiclav) for 6 days
eral practices and
outpatient clinics with placebo for 6 days
[15]
In review Absolute results not reported

Cure rates
[15]
People with clinical- Overall treatment effect OR 1.16
ly diagnosed acute
Systematic 95/124 (77%) with amoxicillin- 95% CI 0.65 to 2.07
rhinosinusitis re-
review clavulanic acid (co-amoxiclav) for
cruited from gener- Not significant
6 days
al practices and
outpatient clinics 93/126 (74%) with placebo for 6
days
Data from 1 RCT

[15]
People with clinical- Cure , at 1 week OR 0.96
ly diagnosed acute
Systematic 36/122 (30%) with amoxicillin- 95% CI 0.56 to 1.65
rhinosinusitis re-
review clavulanic acid (co-amoxiclav) for
cruited from gener- Not significant
6 days
al practices and
outpatient clinics 38/125 (30%) with placebo for 6
days
Data from 1 RCT

-
Adverse effects

-
Ref Results and statistical Effect
(type) Population Outcome, Interventions analysis size Favours
Diarrhoea
[21]
252 people with Diarrhoea , 7 days OR 3.89
clinically diagnosed
RCT with amoxicillin-clavulanic acid 95% CI 2.09 to 7.25
acute rhinosinusitis
(co-amoxiclav) for 6 days
recruited from gen- placebo
eral practices and with placebo for 6 days
outpatient clinics
Absolute results not reported
[15]
In review

[21]
252 people with Diarrhoea , 14 days OR 1.71
clinically diagnosed
RCT with amoxicillin-clavulanic acid 95% CI 0.91 to 3.23
acute rhinosinusitis
(co-amoxiclav) for 6 days
recruited from gen- Not significant
eral practices and with placebo for 6 days
outpatient clinics
Absolute results not reported
[15]
In review

[15]
People with clinical- Serious adverse effects, brain Significance not reported
ly diagnosed acute abscess
Systematic
rhinosinusitis re-
review with amoxicillin-clavulanic acid
cruited from gener-
(co-amoxiclav) for 6 days

© BMJ Publishing Group Ltd 2015. All rights reserved. .......................................................... 10


Ear, nose, and throat disorders
Sinusitis (acute rhinosinusitis)
Ref Results and statistical Effect
(type) Population Outcome, Interventions analysis size Favours
al practices and with placebo for 6 days
outpatient clinics
After 2 weeks of symptomatic
Data from 1 RCT treatment, 1 person who was ini-
tially treated with amoxicillin-
clavulanic acid for 6 days devel-
oped a brain abscess caused by
an amoxicillin-clavulanic acid-
sensitive strain of Streptococcus
milleri; post surgery, the person
was reported to have residual
frontal syndrome

-
-
Cephalosporins or macrolides versus placebo:
We found no RCTs.

-
-
Different treatment durations of cephalosporins versus each other:
[22]
We found one RCT.

-
Improvement in sinusitis
Different treatment durations of cephalosporins compared with each other A 5-day course of cefotiam seems as ef-
fective as a 10-day course of cefotiam at increasing cure rates in people with clinically diagnosed acute rhinosinusitis
(moderate-quality evidence).

Ref Results and statistical Effect


(type) Population Outcome, Interventions analysis size Favours
Cure rates
[22]
1018 people with Cure rate Reported as not significant
clinically diagnosed
RCT 418/489 (85.5%) with 5-day P value not reported
acute rhinosinusitis
course of cefotiam Not significant
in a primary-care
setting 418/490 (85.3%) with 10-day
course of cefotiam

-
Adverse effects

-
Ref Results and statistical Effect
(type) Population Outcome, Interventions analysis size Favours
Adverse effects
[22]
1018 people with Adverse effects
clinically diagnosed
RCT with 5-day course of cefotiam
acute rhinosinusitis
in a primary-care with 10-day course of cefotiam
setting
Absolute results not reported
The RCT reported a low overall
rate of adverse effects (42/1018
[4%])

-
-
Different antibiotics versus each other:
We found no RCTs.

© BMJ Publishing Group Ltd 2015. All rights reserved. .......................................................... 11


Ear, nose, and throat disorders
Sinusitis (acute rhinosinusitis)
-
-
-
Further information on studies
[15]
The systematic review included adults (18 years and older) with acute rhinosinusitis. Ten RCTs were included,
eight of which recruited from primary care settings. All RCTs were in people with mild to moderate acute rhinos-
inusitis; the review found no evidence for people with serious/severe sinusitis (characterised by very high fever,
prolonged symptoms, septic symptoms such as tachycardia, sweating, low blood pressure, or ENT referral
because of serious course of disease or possible complications) or for people with immunosuppression. The
systematic review pooled data to compare antibiotics (any) versus placebo. The results showed that there may
be an increase in overall treatment effect/cure with antibiotics compared to placebo (8 RCTs, 517/813 [64%]
with antibiotics v 459/834 [55%] with placebo, OR 1.25 [95% CI 1.02 to 1.88], P = 0.031). However, the review
highlighted that cure was defined differently in different studies varying from clinical or patient evaluation to
different degrees of change in symptom scores or questionnaire responses. There was no significant difference
in cure at 1 week (4 RCTs, 205/427 [48%] with antibiotics v 198/429 [46%] with placebo, OR 1.07 [95% CI 0.81
to 1.41], P = 0.62) or cure at 2 weeks (3 RCTs, 177/242 [73%] with antibiotics v 144/225 [64%] with placebo,
OR 1.48 [95% CI 0.99 to 2.23], P = 0.055). Purulent secretions were more likely to have resolved at the trial
endpoint with antibiotics (3 RCTs, 236/342 [69%] with antibiotics v 190/318 [60%] with placebo, OR 1.58 [95%
CI 1.13 to 2.22], P = 0.0081). Although it was not possible to pool data for many of the secondary outcomes,
the systematic review found that no individual RCT reported any significant difference in pain duration, illness
duration, or restriction of daily activities. There was, however, an increase in adverse effects with antibiotics
compared to placebo (7 RCTs, 193/706 [27%] with antibiotics v 100/665 [15%] with placebo, OR 2.10 [95% CI
1.60 to 2.77], P <0.00001). The results were also statistically significant for diarrhoea (4 RCTs, 67/421 [16%]
with antibiotics v 41/396 [10%] with placebo, Peto OR 1.81 [95% CI 1.18 to 2.78], P = 0.0061). Only one serious
disease-related adverse effect occurred: one person in the antibiotic group developed a brain abscess. There
were no serious disease-related adverse effects in the placebo group. The systematic review reported that
treatment failure (defined as people having to start on antibiotics due to an abnormal course of rhinosinusitis:
for example, exacerbation, ongoing symptoms, or respiratory complications) was more common in the placebo
group (8 RCTs, 61/1098 [6%] with antibiotics v 115/1077 [11%] with placebo, Peto OR 0.49 [95% CI 0.36 to
0.66], P <0.00001). Six studies in the systematic review received funding from the pharmaceutical industry.
[15]
The systematic review stated that any potential benefits of antibiotics need to be balanced against the risk of
adverse events. The authors concluded that "taking into account antibiotic resistance and the very low incidence
of serious complications ... there is no place for antibiotics for the patient with clinically diagnosed, uncomplicated
acute rhinosinusitis".

-
-
Comment: Clinical guide:
In clinically diagnosed mild to moderate acute rhinosinusitis, there is currently little evidence from
RCTs to support the use of amoxicillin, amoxicillin-clavulanic acid (co-amoxiclav), or doxycycline
over placebo in terms of clinical cure rate. We found no RCTs on the effects of cephalosporins or
macrolides compared with placebo in this group. The risk of adverse effects appears higher with
antibiotics versus placebo, and this should be discussed when considering prescribing antibiotics
for acute rhinosinusitis. In people prescribed a cephalosporin for acute rhinosinusitis, a 5-day course
of cefotiam appears to be as effective as a 10-day course of cefotiam, but the outcome was not
compared with placebo. Antibiotics may still be indicated for severe systemic symptoms in acute
rhinosinusitis, but evidence in this patient group is currently lacking.

OPTION DECONGESTANTS (XYLOMETAZOLINE, PHENYLEPHRINE, PSEUDOEPHEDRINE). . . . . . .

• For GRADE evaluation of interventions for Sinusitis (acute rhinosinusitis), see table, p 15 .
• We found no direct information from RCTs about decongestants (xylometazoline, phenylephrine, pseudoephedrine)
versus placebo, or topical decongestants versus systemic decongestants, in the treatment of people with clinically
diagnosed acute rhinosinusitis.

Benefits and harms


Decongestants versus placebo, or topical decongestants versus systemic decongestants:
We found no systematic review or RCTs.

© BMJ Publishing Group Ltd 2015. All rights reserved. .......................................................... 12


Ear, nose, and throat disorders
Sinusitis (acute rhinosinusitis)
-
-
-
Comment: None.

OPTION SALINE NASAL WASHES VERSUS PLACEBO OR NO TREATMENT. . . . . . . . . . . . . . . . . . . .

• For GRADE evaluation of interventions for Sinusitis (acute rhinosinusitis), see table, p 15 .
• We found no direct information from RCTs about saline nasal washes versus placebo or no treatment in the
treatment of people with clinically diagnosed acute rhinosinusitis.

Benefits and harms


Saline nasal washes versus placebo or no treatment:
We found no systematic review or RCTs.

-
-
-
-
Comment: None.

GLOSSARY
Orbital cellulitis Inflammation of the soft tissues in and around the eye socket.
Rhinorrhoea Discharge from the nasal cavity.
Hyposmia Reduced, although not absent, sense of smell.
Low-quality evidence Further research is very likely to have an important impact on our confidence in the estimate
of effect and is likely to change the estimate.
Moderate-quality evidence Further research is likely to have an important impact on our confidence in the estimate
of effect and may change the estimate.
Very low-quality evidence Any estimate of effect is very uncertain.

SUBSTANTIVE CHANGES
[11]
Corticosteroids (intranasal) versus placebo One systematic review updated. One systematic review reporting
[12] [13]
on a previously included RCT added. One new RCT added. Categorisation unchanged (likely to be beneficial).
Antibiotics (amoxicillin, amoxicillin-clavulanic acid [co-amoxiclav], doxycycline, cephalosporins, macrolides)
[15]
One systematic review added. Existing evidence re-evaluated; categorisation changed to 'unknown effectiveness'.

REFERENCES
1. Benninger MS, Sedory Holzer SE, Lau J. Diagnosis and treatment of uncompli- 10. Ramsey PG, Weymuller EA. Complications of bacterial infection of the ears,
cated acute bacterial rhinosinusitis: summary of the Agency for Health Care paranasal sinuses, and oropharynx in adults. Emerg Med Clin North Am
Policy and Research evidence-based report. Otolaryngol Head Neck Surg 1985;3:143−160.[PubMed]
2000;122:1−7.[PubMed] 11. Zalmanovici A, Yaphe J. Intranasal steroids for acute sinusitis. In: The Cochrane
2. Dubreuil C, Gehanno P, Goldstein F, et al. Treatment of acute maxillary sinusitis Library, Issue 10, 2013. Chichester, UK: John Wiley & Sons, Ltd. Search date
in adult outpatients: comparison of a five versus ten day-course of cefuroxime 2011.
axetil. Med Mal Infect 2001;31:70−78. 12. Hayward G, Heneghan C, Perera R, et al. Intranasal corticosteroids in manage-
3. Kennedy DW. International conference on sinus terminology, staging, therapy. ment of acute sinusitis: a systematic review and meta-analysis. Ann Fam Med
Ann Otol Rhinol Laryngol 1995;104:10. 2012;10:241−249.[PubMed]
4. Jones NS. Rhinosinusitis. In: Statements of clinical effectiveness in otorhinolaryn- 13. Keith PK, Dymek A, Pfaar O, et al. Fluticasone furoate nasal spray reduces
gology. London, UK: British Association of Otorhinolaryngologists, Head and symptoms of uncomplicated acute rhinosinusitis: a randomised placebo-controlled
Neck Surgeons, 1998:21−31. study. Prim Care Respir J 2012;21:267−275.[PubMed]
5. Anon JB, Jacobs MR, Poole MD, et al. Antimicrobial treatment guidelines for 14. Meltzer EO, Bachert C, Staudinger H. Treating acute rhinosinusitis: comparing
acute bacterial rhinosinusitis. Otolaryngol Head Neck Surg 2004;130:1−45. [Er- efficacy and safety of mometasone furoate nasal spray, amoxicillin, and placebo.
ratum in: Otolaryngol Head Neck Surg 2004;130:794−796.][PubMed] J Allergy Clin Immunol 2005;116:1289−1295.[PubMed]
6. Henry DC, Moller DJ Jr, Adelglass J, et al. Comparison of sparfloxacin and clar- 15. Lemiengre MB, van Driel ML, Merenstein D, et al. Antibiotics for clinically diag-
ithromycin in the treatment of acute bacterial maxillary sinusitis. Sparfloxacin nosed acute rhinosinusitis in adults. In: The Cochrane Library, Issue 10, 2013.
Multicenter AMS Study Group. Clin Ther 1999;21:340−352.[PubMed] Chichester, UK: John Wiley & Sons, Ltd. Search date 2012.[PubMed]
7. Low DE, Desrosiers M, McSherry J, et al. A practical guide for the diagnosis and 16. De Sutter AI, De Meyere MJ, Christiaens TC, et al. Does amoxicillin improve
treatment of acute sinusitis. CMAJ 1997;156(suppl 6):S1−S14.[PubMed] outcomes in patients with purulent rhinorrhea? A pragmatic randomized double-
8. De Ferranti SD, Ioannidis JP, Lau J, et al. Are amoxycillin and folate inhibitors blind controlled trial in family practice. J Fam Pract 2002;51:317−323.[PubMed]
as effective as other antibiotics for acute sinusitis? A meta-analysis. BMJ 17. Varonen H, Kunnamo I, Savolainen S, et al. Treatment of acute rhinosinusitis
1998;317:632−637.[PubMed] diagnosed by clinical criteria or ultrasound in primary care: a placebo-controlled
9. Goodman GM, Slavin RG. Medical management in adults of chronic sinus disease. randomised trial. Scand J Prim Health Care 2003;21:121−126.[PubMed]
Immunol Allergy Clin North Am 1994;14:69−87.

© BMJ Publishing Group Ltd 2015. All rights reserved. .......................................................... 13


Ear, nose, and throat disorders
Sinusitis (acute rhinosinusitis)
18. Merenstein D, Whittaker C, Chadwell T, et al. Are antibiotics beneficial for patients trolled double-blind randomized doxycycline trial. Br J Gen Pract
with sinusitis complaints? A randomized double-blind clinical trial. J Fam Pract 1997;47:794−799.[PubMed]
2005;54:144–151.[PubMed] 21. Bucher HC, Tschudi P, Young J, et al. Effect of amoxicillin−clavulanate in clini-
19. Garbutt JM, Banister C, Spitznagel E, et al. Amoxicillin for acute rhinosinusitis: cally diagnosed acute rhinosinusitis: a placebo-controlled, double-blind, random-
a randomized controlled trial. JAMA 2012;307:685−692.[PubMed] ized trial in general practice. Arch Intern Med 2003;163:1793−1798.[PubMed]
20. Stalman W, van Essen GA, van der Graaf Y, et al. The end of antibiotic treatment 22. Gehanno P, Loncle-Provot V, Le Kerneau J. Efficacy of cefotiam hexetil in acute
in adults with acute sinusitis-like complaints in general practice? A placebo-con- maxillary sinusitis, with a short five day vs ten day treatment. Med Mal Infect
2004;34:455−459. [In French][PubMed]

Kim Ah-See
Consultant Otolaryngologist-Head and Neck Surgeon
Aberdeen Royal Infirmary
Aberdeen
UK

Competing interests: KAS declares that he has no competing interests.

Disclaimer

The information contained in this publication is intended for medical professionals. Categories presented in Clinical Evidence indicate a
judgement about the strength of the evidence available to our contributors prior to publication and the relevant importance of benefit and
harms. We rely on our contributors to confirm the accuracy of the information presented and to adhere to describe accepted practices.
Readers should be aware that professionals in the field may have different opinions. Because of this and regular advances in medical research
we strongly recommend that readers' independently verify specified treatments and drugs including manufacturers' guidance. Also, the
categories do not indicate whether a particular treatment is generally appropriate or whether it is suitable for a particular individual. Ultimately
it is the readers' responsibility to make their own professional judgements, so to appropriately advise and treat their patients. To the fullest
extent permitted by law, BMJ Publishing Group Limited and its editors are not responsible for any losses, injury or damage caused to any
person or property (including under contract, by negligence, products liability or otherwise) whether they be direct or indirect, special, inci-
dental or consequential, resulting from the application of the information in this publication.

© BMJ Publishing Group Ltd 2015. All rights reserved. .......................................................... 14


Ear, nose, and throat disorders
Sinusitis (acute rhinosinusitis)
GRADE Evaluation of interventions for Sinusitis (acute rhinosinusitis).
-
Important out-
comes Improvement in sinusitis, Quality of life
Studies (Partici- Type of evi- Consisten-
pants) Outcome Comparison dence Quality cy Directness Effect size GRADE Comment
What are the effects of treatments in people with clinically diagnosed acute rhinosinusitis?
[11]
2 (at least 1450) Improvement in si- Corticosteroids (in- 4 –1 0 –2 0 Very low Quality point deducted for incomplete re-
[14] [13]
nusitis tranasal) versus placebo porting of results in 1 RCT; directness
points deducted for different methods used
to assess symptom improvement and inclu-
sion of people aged under 16 in both stud-
ies
[13]
1 (737) Quality of life Corticosteroids (in- 4 0 0 –2 0 Low Directness points deducted for inclusion of
tranasal) versus placebo people under 16 years and restricted pop-
ulation
[15]
4 (at least 737) Improvement in si- Amoxicillin versus placebo 4 –1 0 –1 0 Low Quality point deducted for incomplete re-
[16] [17] [18] [19]
nusitis porting of results; directness point deduct-
ed for unclear or incomplete outcomes
[15] [19]
1 (155) Quality of life Amoxicillin versus placebo 4 –1 –1 0 0 Low Quality point deducted for sparse data;
consistency point deducted for varying re-
sults at different time points
[15] [17] [20]
2 (284) Improvement in si- Doxycycline versus place- 4 –1 0 –2 0 Very low Quality point deducted for incomplete re-
nusitis bo porting of results; directness points deduct-
ed for use of co-intervention and no statis-
tical analysis between groups in 2 RCTs
[15] [21]
1 (250) Improvement in si- Amoxicillin-clavulanic acid 4 –2 0 0 0 Low Quality points deducted for incomplete re-
nusitis (co-amoxiclav) versus porting of results and some participants
placebo with rhinoscopy
[22]
1 (979) Improvement in si- Different treatment dura- 4 –1 0 0 0 Moderate Quality point deducted for incomplete re-
nusitis tions of cephalosporins porting of results
versus each other
We initially allocate 4 points to evidence from RCTs, and 2 points to evidence from observational studies. To attain the final GRADE score for a given comparison, points are deducted or added from this initial
score based on preset criteria relating to the categories of quality, directness, consistency, and effect size. Quality: based on issues affecting methodological rigour (e.g., incomplete reporting of results, quasi-
randomisation, sparse data [<200 people in the analysis]). Consistency: based on similarity of results across studies. Directness: based on generalisability of population or outcomes. Effect size: based on magnitude
of effect as measured by statistics such as relative risk, odds ratio, or hazard ratio.

© BMJ Publishing Group Ltd 2015. All rights reserved. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

You might also like