Crop Protection 67 (2015) 26e34

Contents lists available at ScienceDirect

Crop Protection
journal homepage: www.elsevier.com/locate/cropro

Standard area diagrams for assessment of powdery mildew severity

on tomato leaves and leaflets
Daniel A. Costa Lage a, 1, Waldir A. Marouelli b, Henrique da S. S. Duarte c,

-Filho a, *
Adalberto C. Cafe
a
Departamento de Fitopatologia, Universidade de Brasília, 70910-900, Brasília, DF, Brazil
b
Embrapa Hortaliças, 70359-970, Brasília, DF, Brazil
c
Departamento de Proteça~o de Plantas, Universidade Federal do Parana
, 80035-050, Curitiba, PR, Brazil

a r t i c l e i n f o a b s t r a c t

Article history: Two standard area diagrams (SADs) to assess the severity of powdery mildew (Leveillula taurica) on
Received 30 May 2014 tomato leaves and leaflets were developed and evaluated. The SADs are composed of two sets of images
Received in revised form of leaves and leaflets with six distinct percentages of diseased areas (1, 5, 10, 20, 40 and 60%). Diagrams
19 September 2014
were evaluated by 16 raters divided into two groups, based on their former experience in assessing plant
Accepted 20 September 2014
Available online
disease intensities, using the two sets of images, first without SADs and then using the proposed SADs.
Lin's concordance correlation analysis of estimated vs. actual disease severities showed that precision
and accuracy were clearly improved when raters were aided by the proposed diagrams, for both leaflets
Keywords:
Solanum lycopersicum
and leaves. Based on coefficients of determinations (R2) and on the intra-class correlations (r), the es-
Tomato powdery mildew timates of severity were consistently more reliable when using SADs. Furthermore, when the scales were
Oidiopsis haplophylli adopted, the performance of the inexperienced group improved to the same level of that of the expe-
Disease quantification rienced group. The diagrams improved subjects' ability to accurately, precisely and reliably estimate
Epidemiology tomato powdery mildew severity, and as such can be used to assess severity for studies in epidemiology,
quantitative host resistance and in the evaluation of management practices in this important
pathosystem.
© 2014 Elsevier Ltd. All rights reserved.

1. Introduction However, their use alone or in combination with one another

Tomato (Solanum lycopersicum L.) is widely cultivated
throughout the world, but the crop is challenged by a large
number of plant diseases, which strongly compromise yield (Jones
et al., 1991). The use of chemicals has been the primary manage-
ment practice to reduce disease risk, but other management
practices are also available (Drinkwater et al., 1995). A basic factor
that affects disease intensity in most foliar pathosystems is leaf
wetness (Rotem and Palti, 1969), and among the agronomic
practices aimed at reducing it are the choice of furrow or drip
irrigation, dry season cropping and cultivation in protected envi-
ronments (Diver et al., 1999). These measures hinder infection
efficiency of the majority of foliar pathogens and have been
increasingly adopted by tomato growers (e.g. Cabral et al., 2013).
* Corresponding author. Tel.: þ55 61 31072917; fax: þ55 61 31072931.
E-mail addresses: cafefilh@unb.br, adalberto.cafe@pesquisador.cnpq.br

-Filho).
(A.C. Cafe
1
Present address: Bayer CropScience e SHIS QI 07 Bloco G Loja 102-B e Caixa
Postal 7048, 71645-970, Brasília, DF, Brazil.
creates an ideal environment for severe epidemics of powdery
mildew (Reis et al., 2007), due to the fact that the presence of leaf
wetness actually reduces infection by the Erysiphaceae, the causal
agents of powdery mildew (Sutton and Jones, 1979). Powdery
mildews constitute an important group of fungal phytopathogens
that are adapted to drier environments and are widely distributed
throughout the world.
Tomato is host to more than one species that causes powdery
mildew, including the epiphytic anamorphic forms Oidium lyco-
persici Cook & Massee emend. and Oidium neolycopersici Kiss (Kiss

v.) Arnaud (anamorph,
et al., 2001). However, Leveillula taurica (Le
Oidiopsis haplophylli Rulamort) is the most prevalent and
economically damaging powdery mildew agent to tomato (Correll
et al., 1988). L. taurica is an important pathogen on the crop in
dry conditions, when air relative humidity (RH) is between 50 and
70%, and temperatures are between 20 and 25
C (Guzman-Plazola
et al., 2003; Palti, 1988).
The symptoms caused by L. taurica are distinct from those
usually associated with other members of the Erysiphaceae that

http://dx.doi.org/10.1016/j.cropro.2014.09.014
0261-2194/© 2014 Elsevier Ltd. All rights reserved.
 D.A. Costa Lage et al. / Crop Protection 67 (2015) 26e34
cause powdery mildew. In the tomato plant, symptoms are initi-
ated by small chlorotic lesions that turn yellowish and become
visible on the adaxial surface of the leaf, which corresponds with
the colonization of the fungus on the abaxial surface and profuse
sporulation (Reis et al., 2005). Under favorable conditions, the
inoculum generated in successive cycles of the disease, results in
rapid increase of the affected leaf area. Old lesions become
necrotic and are generally surrounded by chlorotic leaf tissue
(Palti, 1988). Losses are further compounded when fruit yield of
severely affected plants is lost by sun scald (Jones and Thomson,
1987). Under favorable conditions, the estimated fruit yield los-
ses due to Leveillula powdery mildew may reach 31% (Jones and
Thomson, 1987). In dry regions or in the greenhouse, the path-
ogen presents a constant threat to growers, and a pressing need
for epidemiological and management studies to reduce disease
risk.
So that quantitative studies on the epidemiology and man-
agement of powdery mildew of tomato may achieve the highest
possible quality in terms of accuracy and precision, it is essential
to have accurate, precise and reliable disease estimates. The ac-
curacy of an estimate has been defined as how close the estimate
is to the true value (Nutter and Schultz, 1995), while precision is
defined by the variability of the estimates (Yadav et al., 2013),
and reliability is associated to the extent to which the same es-
timate obtained under different conditions yields similar results
(Nutter et al., 1991). In order to compare estimated and actual
values, Madden et al. (2007) discussed the concept of agreement,
defined as the product of precision and accuracy. Several
methods exist for determining and improving the precision, ac-
curacy and reliability of disease assessments (Bock et al., 2010;
Madden et al., 2007; Nutter et al., 1993), including the use of
standard area diagrams (SADs). SADs are illustrations of diseased
plant parts, within a range of severity values that are used as
references to which to compare samples when estimating dis-
ease severity. For this, representative symptoms of the disease
Fig. 1. Standard area diagrams for powdery mildew (Leveillula taurica) severity on tomato (Solanum lycopersicum L.) leaflets. The symptoms of chlorosis and yellowing are rep-
resented in light gray, while the symptoms of necrosis are shown in dark gray. The numbers represent actual percentage of the leaflet area showing the sum of both chlorotic and
necrotic symptoms of the disease.
27
should be identified and the disease severity in the field must be
ascertained. Furthermore, the SAD must be quantitatively eval-
uated before being recommended as a tool for improving disease
severity estimates.
However, despite the economic importance of the tomato crop
and the high losses incurred by powdery mildew, so far no SAD to
evaluate Leveillula on tomato has been proposed. Both the
complexity of the symptoms, which evolves over time and com-
prises leaf chlorosis and necrosis, as well as the complex foliar
morphology of the tomato plant, may partially explain why no such
scale exists for this pathosystem. None of the most up-to-date
studies that discuss epidemiology and control of tomato powdery
mildew (Guzma
n-Plazola et al., 2011; Kasselaki et al., 2006;
Konstantinidou-Doltsinis et al., 2006) mention the use of dia-
grams as a tool for evaluating disease severity.
The goal of this study was to develop and evaluate a set of SADs
encompassing the full range of disease severities as found in the
field, as an aid to raters estimating tomato powdery mildew,
bearing in mind the tomato plant's leaf anatomy, as well as the
disease pathogenesis and the symptoms of chlorosis and necrosis.
2. Materials and methods
2.1. Development of the SAD sets to assess the severity of powdery
mildew on tomato leaves and leaflets
Two hundred naturally field-infected tomato leaflets and leaves,
that encompassed the widest possible range of disease severities
were collected, during two tomato cropping seasons in Mid-West
Brazil. Images of each leaflet and leaf were captured digitally
against a black background using a Hewlett Packard Photosmart
scanner (Model C3180, Sa~o Paulo, Brazil) with an image resolution
of 300 dpi, and individually processed with the aid of Quant soft-
ware version 1.0.2 (Vale et al., 2003). The necrotic and chlorotic
diseased areas were measured separately and the proportion of
28 D.A. Costa Lage et al. / Crop Protection 67 (2015) 26e34
diseased leaf or leaflet tissue was estimated using Quant. The
diseased area determined electronically was considered the actual
disease severity and used as the reference for evaluating the ac-
curacy and precision of assessments performed with and without
the SADs. Representative images were used to develop two SAD
sets, one for the leaves and the other for leaflets. Each image on the
diagram represents an actual image vectorized in graphic software
(CorelDRAW version X4), seeking to maintain the actual form and
size of the lesions. Symptoms of chlorosis were represented in light
gray, while symptoms of necrosis were shown in dark gray.
2.2. Validation of the SAD sets to assess the severity of powdery
mildew on tomato leaflets and leaves
In order to validate the diagram sets to tomato, 16 subjects were
recruited, divided into two equal groups of inexperienced and
experienced raters, according to their knowledge of disease quan-
tification. The group of inexperienced raters was composed of eight
undergraduates who had never assessed severity of plant diseases,
while the group of experienced raters was made up of eight post-
graduate students familiar with disease severity assessments.
Each of the SADs was validated at separate times, but followed the
same procedures. The first stage involved each rater assessing the
severity of powdery mildew of 42 randomly-presented images of
tomato leaves or leaflets with various levels of the disease. Later,
the same images were presented for disease estimation, in a
different random order, and raters were again asked to assess dis-
ease severity with the help of the proposed diagrams.
Accuracy and precision (agreement) of the estimates by each
rater, with and without the use of the SADs, was determined
based on Lin's concordance correlation coefficient (LCCC, rc) (Lin,
1989), as described by Nita et al. (2003). The LCCC combines
measures of accuracy and precision to assess the relational fit of
the data to the line of concordance (45
), and is defined as:
rc ¼ r  Cb, where Cb is a bias correction factor that measures
how far the best-fitting line deviates from 45
and is thus a
measure of accuracy, and r is the correlation coefficient between
Fig. 2. Standard area diagrams for powdery mildew (Leveillula taurica) severity on tomato (Solanum lycopersicum L.) leaves. The symptoms of chlorosis and yellowing are repre-
sented in light gray, while the symptoms of necrosis are shown in dark gray. The numbers represent actual percentage of the leaf area showing the sum of both chlorotic and
necrotic symptoms of the disease.
estimated severity (Y) and actual severity (X), which measures
precision (variation) or the scattering of points around the best-
fitting line. Cb is estimated as: Cb ¼ 2/[(y þ 1/y þ u2)], where
y ¼ sy/sx, and sy and sx are the standard deviations of Y and X,
pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
respectively; and y ¼ ðmy  mx Þ= ðsy $sx Þ, where my and mx are the
mean values of Y and X, respectively. The term y measures the
difference in slope between the fitted line and the line of
concordance (scale bias). Equal slopes would have a y of 1. The
term u is a measure of the location shift reflecting height dif-
ferences between the fitted line and the line of concordance
(location bias). Equal heights would have a u of 0. When there is
perfect agreement between estimated and actual severity, the
points fall on the concordance line and my ¼ mx, sy ¼sx, r ¼ 1, and
by definition, y ¼ 1, u ¼ 0, Cb ¼ 1, and rc ¼ 1 (Bock et al., 2010;
Nita et al., 2003).
Inter-rater reliability of the estimates was determined in two
ways. First, using the coefficient of determination (R2) from linear
regression analyses of relationships between severity estimates for
all pairs of raters (Nutter and Schultz, 1995). Second, using the
intracluster (or intra-class) correlation (r) for all raters combined,
as described by Shoukri and Pause (1999) and Nita et al. (2003).
Precision was also determined with analysis of the absolute error
(estimated severity minus actual severity).
For all parameters analyzed (r, Cb, y, u, and rc) and for the inter-
rater reliability (R2), the differences between means (i.e. with SADs
minus without SADs) were calculated and submitted to an equiv-
alence test for significance (Bardsley and Ngugi, 2013; Yadav et al.,
2013; Yi et al., 2008). The equivalence test was used to calculate the
95% confidence intervals (CIs) for each statistic (the difference be-
tween the means) by bootstrapping using the percentile method
(with an equivalence test, the null hypothesis is the converse of H0,
i.e. the null hypothesis is non-equivalence). All analyses were based
on 2000 balanced bootstrap samples using PROC SURVEY SELECT
and calculating 95% CI using PROC UNIVARIATE (SAS Institute Inc.,
Cary, NC, USA). If the CIs embraced zero, the difference was
considered non-significant (a ¼ 0.05).
 D.A. Costa Lage et al. / Crop Protection 67 (2015) 26e34 29

No SAD SAD used

100 100
A B
B
80 80

60 60

40 40

20 20

0 0

100 100
C D
80 80

60 60

40 40
Estimatedseverity(%)

20 20

0 0

100 100
E F
80 80

60 60

40 40

20 20

0 0

100 100
G H
80 80

60 60

40 40

20 20

0 0
0 20 40 60 80 100 0 20 40 60 80 100

Actual severity (%)

Fig. 3. Relationship between actual and estimated severity of powdery mildew (Leveillula taurica) of tomato (Solanum lycopersicum L.) without and with the use of a set of standard
area diagrams (SADs) for 42 diseased leaflets (A, B, C and D) and 42 diseased leaves (E, F, G and H). A, B, E and F represent the 8 raters with experience and C, D, G and H represent
the 8 raters without experience in disease severity assessment. The solid line represents the best-fitting line, whereas the dotted line is the concordance line, which represents
perfect agreement between actual and estimated severity (slope of 1, intercept of 0).

3. Results Fig. 3 B, D). Accuracy, precision and agreement were generally

3.1. Standard area diagram set to assess the severity of powdery
mildew on tomato leaflets
3.1.1. Effect of leaflet SADs on accuracy and precision
The SAD set designed in this study at the leaflet level has six
leaflet images, with powdery mildew severities from 1 to 60%
(Fig. 1), based on the range of severities observed occurring natu-
rally in tomato fields (0.5e65%).
Visual assessments of severity of powdery mildew on leaflets
were closer to the actual values when the diagrams were
employed, compared to unaided visual assessments (Fig. 3 A, C vs.
improved. Although the improvement was slight in the location
bias (Table 1, u ¼ 0.14 and 0.03 without and with SADs, respec-
tively), the statistical parameters r, Cb, y, and rc were all signifi-
cantly improved when raters used the diagrams to estimate
disease severity. The scale bias (y) was 1.26 and 1.05 without and
with SADs, respectively and the correction factor (Cb) was 0.88 and
0.98 without and with SADs, respectively (Table 1). When SADs
were not employed, the rc was 0.73 and when the SADs were used,
the rc significantly improved to 0.88. Without use of the SADs, the
precision (r) was 0.82, but when the SADs were employed, pre-
cision was 0.90 (Table 1). The more precise evaluation of powdery
mildew severity using the SADs was also confirmed by reduction
30 D.A. Costa Lage et al. / Crop Protection 67 (2015) 26e34

in absolute error, with most falling from c. ±40% (Fig. 4 A, C) to c.
±20% (Fig. 4 B, D).
3.1.2. Rater experience on accuracy and precision with leaflet SADs
As expected, without the aid of the diagrams, experienced raters
estimated powdery mildew severity significantly better than
inexperienced raters for almost all statistical parameters (Cb, y, u
and rc, Table 2), except for precision, which did not differ between
groups (r ¼ 0.81 and 0.83). On the other hand, with aid of the SADs,
almost all statistical parameters (except u) were not significantly
different between groups of inexperienced and experienced raters
(Table 2). Thus, the use of the SADs improved accuracy and preci-
sion of the estimates of powdery mildew severity by inexperienced
raters to almost the same level as experienced raters.
Use of the diagrams was associated to a tendency to over-
estimate disease severity by the inexperienced raters. This was
made evident by the fact that 85.7% of the inexperienced raters
showed positive location parameter (u) values (data not shown).
Moreover, this trend was further confirmed by the fact that positive
absolute errors were detected for most estimates (Fig. 4B).
Contrarily, experienced raters using SADs were more disposed to
underestimate disease severity (75% of the experienced raters
showed negatives u values, data not shown). This trend was also
confirmed by the fact that negative absolute errors were detected
for most estimates (Fig. 4D).
3.1.3. Inter-rater reliability
Based on the equivalence test, the statistical parameters of the
intra-class correlation coefficient (r) and the inter-rater coefficient
of determination (R2) were significantly enhanced when raters
used the SADs to estimate powdery mildew severity on leaflets.
With aid of the diagrams, r advanced from r ¼ 0.689 to r ¼ 0.868
(Table 3). Similarly, the R2 mean of the pairwise comparisons
increased from 0.83 to 0.89 (Table 3), indicating that use of the
diagrams improved inter-rater reliability.
3.2. Standard area diagram set to assess the severity of powdery
mildew on tomato leaves
3.2.1. Effect of leaf SADs on accuracy and precision
The SAD set designed at the whole leaf level has six leaf images,
each with a distinct severity of powdery mildew from 1 to 60%
(Fig. 2), based on the observed range of severities observed in to-
mato fields, which ranged from 0.5 to 62%.
Assessments were closer to the actual values when SADs
were employed (Fig. 3 E, F, G, H). The accuracy, the precision
and the agreement of perceived and actual powdery mildew
assessments on leaves were improved. There was a slight
improvement in the scale bias (y ¼ 1.06 and 0.99 without
and with SADs, respectively), but all other parameters (r, Cb, u,
and rc) were significantly improved when the diagrams were
employed. The location bias (u) was 0.12 and 0.05 without and
with SADs, respectively. The correction factor (Cb) was 0.92 and
0.98 without and with SADs, respectively. When SADs were not
employed, the rc was 0.81 and when the SADs were used, the rc
increased to 0.91. Without use of the SADs, the precision (r)
was 0.88, but when the SADs were employed, precision
was 0.93 (Table 1). The more precise evaluation of powdery
mildew severity using the SADs was also confirmed by
reduction in absolute error, with most falling within ±20% (Fig. 4
E, F, G, H).
3.2.2. Rater experience on accuracy and precision with leaf SADs
As anticipated, without the visual aid of the SADs, the group of
experienced raters had significantly better agreement between
estimated and actual severity values than the group of inexperi-
enced raters, as determined by the Lin's concordance correlation
coefficient (rc, Table 2). Conversely, with the aid of the diagrams, no
differences were detected in the LCCC (rc) between the two groups,
and only the scale and location parameters (y and u) remained
significantly different between the group of experienced and
inexperienced raters (Table 2). Therefore, adoption of the diagrams
as visual aid to estimate tomato powdery mildew severity on leaves
improved the performance of inexperienced raters to almost the
same level as experienced raters.
In a similar manner to what was observed with the leaflet SADs,
there was a tendency to overestimate disease severity on leaves by
inexperienced raters and 62.5% of the inexperienced raters showed
positive location parameter (u) values (data not shown). This is
confirmed by the fact that positive absolute errors were detected
for most estimates (Fig. 4F). On the other hand, experienced raters
using SADs tended to underestimate disease severity, and 62.5%
had negative u values (data not shown). This trend was further

Table 1
Effect of using a standard area diagram set (SADs) as an assessment aid on the bias, accuracy, precision and agreement of assessments of severity of powdery mildew on 42
tomato leaflets and leaves as estimated by16 raters.

Diagram Parameters Meansa Differenceb between means 95% CIsc of the difference

W/o SAD With SAD

Leaflet diagram set Scale (y)d 1.26 (0.25) 1.05 (0.10) 0.21 (0.06) ¡0.320 to ¡0.109
Location (u)e 0.14 (0.47) 0.03 (0.19) 0.12 (0.10) 0.312 to 0.062
Coefficient of bias (Cb)f 0.88 (0.12) 0.98 (0.03) 0.10 (0.03) 0.049e0.155
Correlation coefficient (r)g 0.82 (0.05) 0.90 (0.04) 0.08 (0.01) 0.055e0.098
LCCC (rc)h 0.73 (0.11) 0.88 (0.05) 0.16 (0.02) 0.117e0.199
Leaf diagram set Scale (y) 1.06 (0.27) 0.99 (0.08) 0.07 (0.06) 0.195 to 0.022
Location (u) 0.12 (0.39) 0.05 (0.19) 0.18 (0.07) ¡0.318 to ¡0.042
Coefficient of bias (Cb) 0.92 (0.10) 0.98 (0.02) 0.06 (0.02) 0.019e0.113
Correlation coefficient (r) 0.88 (0.04) 0.93 (0.02) 0.05 (0.01) 0.033e0.064
LCCC (rc) 0.81 (0.09) 0.91 (0.03) 0.10 (0.02) 0.070e0.144
a
The values for standard deviation are in parentheses.
b
Mean of the difference between each rating. The values for standard errors are in parentheses (bootstrap calculated values).
c
2000 bootstrap samples were used to obtain the confidence intervals (CIs). If the CIs embrace zero, the difference was not significant (a ¼ 0.05). Bold numbers represent
significance of the difference.
d
Scale bias or slope shift (y, 1 ¼ no bias relative to the concordance line).
e
Location bias or height shift (u, 0 ¼ no bias relative to the concordance line).


f
The correction factor (Cb) measures how far the best-fit line deviates from 45 and is a way to measure accuracy.
g
The precision is measured by the correlation coefficient (r).
h
Lin's concordance correlation coefficient (LCCC) combines both measures of precision (r) and accuracy (Cb) to measure agreement with the true value.
 D.A. Costa Lage et al. / Crop Protection 67 (2015) 26e34 31

confirmed by the fact that negative absolute errors were detected
for most estimates by experienced raters (Fig. 4H).
3.2.3. Inter-rater reliability
Both the intra-class correlation coefficient (r) and the coefficient
of determination (R2) were significantly improved when the raters
used the SADs to estimate powdery mildew severity on the leaves.
The r value was greater when the SADs were used (r ¼ 0.717 un-
aided vs. r ¼ 0.875 with SADs). Without the aid of the SADs, the R2
mean of the pairwise comparisons was 0.85, but when the dia-
grams were employed, the R2 mean increased to 0.90. All param-
eters thus indicate that use of the leaf SADs provided greater inter-
rater reliability (Table 3).
4. Discussion
Standard area diagrams are important for quantification of
disease severity to be used in epidemiological studies, screening
for partial resistance and assessment of disease management
schemes. Results demonstrated that SADs improve accuracy,
precision and reliability of the tomato powdery mildew data
collection and will help minimize the risk of erroneous decisions
or Type II errors in statistical analysis (Bock et al., 2010; Nutter
et al., 1993). SADs have been demonstrated to improve the accu-
racy and reliability of assessment of diseases in different crops
including corn white spot (Capucho et al., 2010), potato early
blight (Duarte et al., 2013), wheat blast (Rios et al., 2013), pecan

No SAD SAD used

60 60
A
A B
B
40 40

20 20

0 0

-20 -20

-40 -40

-60 -60

60 60
CC DD
40 40

20 20

0 0

-20 -20
Absolute error

-40 -40

-60 -60

60 60
AE B
F
40 40

20 20

0 0

-20 -20

-40 -40

-60 -60

60 60
C
G DH
40 40

20 20

0 0

-20 -20

-40 -40

-60 -60
0 10 20 30 40 50 60 0 10 20 30 40 50 60

Actual severity (%)

Fig. 4. Absolute error (estimated severity minus actual severity) of estimates of severity of powdery mildew (Leveillula taurica) on tomato (Solanum lycopersicum L.) without and
with the use of a set of standard area diagrams (SADs) for 42 diseased leaflets (A, B, C and D) and 42 diseased leaves (E, F, G and H). A, B, E and F represent 8 raters with experience
and C, D, G and H represent 8 raters without experience in disease severity assessment. Low absolute errors indicate that the estimated severity was similar to the actual severity.
32 D.A. Costa Lage et al. / Crop Protection 67 (2015) 26e34

Table 2
Effect of rater experience without and with the use of standard area diagram sets (SADs) as an assessment aid on the bias, accuracy, precision and agreement of assessments of
severity of powdery mildew on 42 tomato leaflets and leaves as estimated by16 raters.

Diagram SADs Parameters Means of experiencea Differenceb between means 95% CIsc of the difference

Inexperienced (n ¼ 8) Experienced (n ¼ 8)

Leaflet diagram set W/o SAD Scale (y)

d
1.41 (0.24) 1.10 (0.16) 0.31 (0.10) ¡0.484 to ¡0.095
Location (u)e 0.45 (0.46) 0.17 (0.18) 0.57 (0.22) ¡0.943 to ¡0.090
Coefficient of bias (Cb)f 0.81 (0.13) 0.96 (0.02) 0.15 (0.04) 0.071¡0.238
Correlation coefficient (r)g 0.81 (0.06) 0.83 (0.11) 0.01 (0.03) 0.035 to 0.062
LCCC (rc)h 0.66 (0.11) 0.80 (0.06) 0.14 (0.04) 0.065¡0.205
With SAD Scale (y) 1.08 (0.11) 1.02 (0.08) 0.05 (0.04) 0.131 to 0.019
Location (u) 0.13 (0.18) 0.08 (0.13) 0.21 (0.07) ¡0.341 to 0.093
Coefficient of bias (Cb) 0.97 (0.04) 0.99 (0.01) 0.01 (0.01) 0.007 to 0.442
Correlation coefficient (r) 0.89 (0.04) 0.92 (0.03) 0.03 (0.02) 0.001 to 0.062
LCCC (rc) 0.86 (0.05) 0.90 (0.03) 0.03 (0.02) 0.011 to 0.068
Leaf diagram set W/o SAD Scale (y) 1.13 (0.37) 0.99 (0.08) 0.14 (0.12) 0.396 to 0.070
Location (u) 0.30 (0.43) 0.06 (0.27) 0.37 (0.17) 0.661 to 0.008
Coefficient of bias (Cb) 0.87 (0.12) 0.97 (0.03) 0.09 (0.05) 0.004¡0.192
Correlation coefficient (r) 0.89 (0.04) 0.88 (0.04) 0.01 (0.02) 0.039 to 0.029
LCCC (rc) 0.77 (0.11) 0.85 (0.03) 0.08 (0.04) 0.014¡0.156
With SAD Scale (y) 1.03 (0.10) 0.95 (0.04) 0.07 (0.03) ¡0.129 to ¡0.008
Location (u) 0.05 (0.17) 0.16 (0.16) 0.20 (0.08) ¡0.361 to 0.065
Coefficient of bias (Cb) 0.98 (0.02) 0.98 (0.02) 0.01 (0.01) 0.022 to 0.012
Correlation coefficient (r) 0.92 (0.02) 0.94 (0.02) 0.02 (0.01) 0.001 to 0.034
LCCC (rc) 0.91 (0.04) 0.92 (0.02) 0.011 (0.01) 0.015 to 0.037
a
The values for standard deviation are in parentheses.
b
Mean of the difference between each rating. The values for standard errors are in parentheses (bootstrap calculated values).
c
2000 bootstrap samples were used to obtain the confidence intervals (CIs). If the CIs embrace zero, the difference was not significant (a ¼ 0.05). Bold numbers represent
significance of the difference.
d
Scale bias or slope shift (y, 1 ¼ no bias relative to the concordance line).
e
Location bias or height shift (u, 0 ¼ no bias relative to the concordance line).


f
The correction factor (Cb) measures how far the best-fit line deviates from 45 and is a way to measure accuracy.
g
The precision is measured by the correlation coefficient (r).
h
Lin's concordance correlation coefficient (LCCC) combines both measures of precision (r) and accuracy (Cb) to measure agreement with the true value.

scab (Yadav et al., 2013) and pepper anthracnose (Pedroso et al., mildew observed in the field. Too few diagrams in one set can
2011). compromise accuracy and precision, while an excessive number of
Powdery mildew is a disease that leads to heavy damage in diagrams can be time consuming and affect the efficiency of as-
tomato production. Accurate assessments of actual disease se- sessments (Madden et al., 2007). Correa et al. (2009) have shown
verities are paramount for implementing powdery mildew yield that six diagrams are sufficient for tomato leaf blight and more than
loss estimates, supporting breeding programs, and testing the ten diagrams are excessive and result in loss of time.
efficacy of various control methods, among other studies. How-
ever, the variation in symptoms, along with the composite anat-
Table 3
omy of the tomato leaf, may lead to reduced accuracy and
Inter-rater reliability of assessments by 16 raters of powdery mildew on 42 leaflets
precision in the estimates. These were the main problems high- and leaves of tomato both without and with the use of a standard area diagram set
lighted by raters who helped to evaluate the diagrammatic scales (SADs) assessment aid. Inter-rater reliability is measured by the intra-class corre-
that we developed. Furthermore, the diagrammatic scales lation coefficient (r) and coefficient of determination (R2).
currently available for evaluating other leaf diseases in tomato do Diagram Statistics Without aid With aid of SADs
not realistically represent symptoms of powdery mildew on to- of SADs
mato leaves and/or leaflets, which justified the need to develop Leaflet Intra-class 0.689 F, P > Fa: 0.868 F, P > F:
specific scales for this disease. diagram correlation L ¼ 75 L ¼ 132
The two scales for assessing disease in leaflets and leaves are set coefficient (r) (P < 0.0001); (P < 0.0001);
designed for use in different approaches to study tomato powdery R ¼ 46 R ¼ 12
(P < 0.0001) (P < 0.0001)
mildew. Currently, it is common to find published reports in which
Mean inter-rater 0.83 (0.62e0.94) 0.89 (0.81e0.95)
several tomato diseases, including the L. taurica mildew, are studied coefficient of Mean differencec ¼ 0.065 (0.0002),
at leaflet level, as in host resistance and plantepathogen in- determination (R2)b 95% CIs 0.047 e 0.086
teractions in controlled conditions (e.g. Guzman-Plazola et al., Leaf Intra-class correlation 0.717 F, 0.875 F,
2003). In these types of experiments the leaflet diagrams would diagram coefficient (r) P > Fa: L ¼ 75 P > F: L ¼ 151
set (P < 0.0001); (P < 0.0001);
be useful, because often only leaflets are inoculated or the assay is R ¼ 36 R ¼ 16 (P < 0.0001)
conducted in detached leaflets. On the other hand, the use of whole (P < 0.0001)
leaves for severity assessment is also common, as in comparison of Mean inter-rater 0.85 (0.72e0.94) 0.90 (0.83e0.98)
fungicide effects, host partial resistance or other management coefficient Mean differencec ¼ 0.051 (0.0001),

n-Plazola et al., 2011). The proposed leaf di- of determination (R2)b 95% CIs 0.037 e 0.066
strategies (e.g. Guzma
a
agrams will be useful when raters wish to collect severity data on F-value for L ¼ leaf, R ¼ rater. F e value in parentheses.
b
whole leaves or at the plant level. Mean coefficients of determination estimated from pairwise comparisons of
assessments by all visual raters.
The two SADs proposed in this study have six images with a c
Mean of the difference between each rating, with standard errors in parentheses
range of severity distributed from 1 to 60%. The number of diagrams (bootstrap calculated value), confidence intervals (CIs) were based on 2000 boot-
is quite sufficient to represent the range of intensities of powdery strap samples. If the CIs embrace zero, the difference is not significant (a ¼ 0.05).
 D.A. Costa Lage et al. / Crop Protection 67 (2015) 26e34
Without aid of SADs, there was a significant effect of rater
experience on rc (which combines the measures of both precision
and accuracy), where experienced raters were more accurate and
precise. However, there were no significant differences on rc when
the experienced and inexperienced groups rated severity using the
SADs. Therefore, performance of inexperienced raters improved up
to a point where no significant differences were detected between
the two groups. These effects have been observed in previous
studies (e.g., Gonz
alez-Domínguez et al., 2014; Pedroso et al., 2011;
Yadav et al., 2013). Overall, results showed that both of the SAD sets
developed and validated in this study improved the accuracy,
precision and reliability of disease severity estimates, and are good
tools for quantifying powdery mildew severity on leaflets and
leaves.
Even as absolute errors were consistently reduced, a tendency
to overestimate disease severity by inexperienced raters and a
tendency to underestimate disease severity by inexperienced
raters using SADs was observed in this study. According to Nutter
and Schultz (1995), solutions to correct or minimize inaccurate
disease severity assessments vary according to the magnitude of
the error and the amount of training received by the rater.
Computer-based training of raters is one valuable approach to
reduce errors and increase the accuracy of tomato powdery
mildew severity estimates, as suggested by several authors
(Bardsley and Ngugi, 2013; Nutter and Schultz, 1995; Nutter and
Worawitlikit, 1989).
One of the most commonly used approaches for validating
disease assessment methods and estimating the accuracy and
precision of estimates is linear regression (Bock et al., 2010;
Capucho et al., 2010; Nutter et al., 1993; Nutter and Schultz,
1995; Nutter and Esker, 2006). We have indeed resorted to
linear regression to demonstrate the relationship between actual
and estimated severity in this paper (Fig. 3). However, there has
been an argument that erroneous conclusions may be drawn from
linear regression analysis under certain circumstances (Lin, 1989;
Madden et al., 2007). Linear regression analysis may not detect
departure from intercept 0 and slope 1 if the data are very scat-
tered. The less precise the data, the less likely the null hypothesis
(b0 ¼ 0 and b1 ¼ 1) will be rejected, and conversely, if the system is
highly reproducible, the hypothesis could be rejected due to very
small error (Lin, 1989). Thus, we have also employed Lin's
concordance correlation because it combines measures of accu-
racy and precision and avoids these problems associated with
linear regression. In the present study, accuracy, precision and
reliability of SADs to estimate powdery mildew severity were
determined using an equivalence test, recommended for agree-
ment studies (Yi et al., 2008). The equivalence test has been used
in the analysis of other disease diagrams to statistically estimate
agreement and reliability (Bardsley and Ngugi, 2013; Yadav et al.,
2013).
While SADs are important for the quantitative appraisal of the
efficacy of disease management practices, they are not designed for
real-time management decisions, such as time of fungicide appli-
cations. Because SADs are based on symptoms, and not on pathogen
signs, they do not adequately measure latent periods. In fact, in
some specific environmental conditions, sporulation ahead of
chlorosis may be detected in some tomato genotypes and the use of
the diagrams alone will not account for this important epidemio-
logical variable.
In summary, the standard area diagrams proposed in this study
consistently and significantly improved the accuracy, precision and
reliability of the estimates of L. taurica powdery mildew severity on
tomato leaflets and leaves. It is thus hoped that these SADs will
prove to be useful tools for quantitative studies in epidemiology
and disease management in this important pathosystem.
33
Acknowledgments
We gratefully acknowledge Dr. Emerson Del Ponte (UFV) for the
review of the statistical analysis and valuable discussions. The first
author was partially supported by a doctorate sholarship of the
Brazilian National Research Council (CNPq). AC Cafe
-Filho and WA
Marouelli are recipients of CNPq Research Fellowships, grant
numbers 305792/2013-0 and 302954/2010-4.
References
Bardsley, S.J., Ngugi, H.K., 2013. Reliability and accuracy of visual methods used to
quantify severity of foliar bacterial spot symptoms on peach and nectarine.
Plant Pathol. 62, 460e474.
Bock, C.H., Poole, G., Parker, P.E., Gottwald, T.R., 2010. Plant disease severity esti-
mated visually, by digital photography and image analysis, and by hyperspectral
imaging. Crit. Rev. Plant Sci. 29, 59e107.
Capucho, A.S., Zambolim, L., Duarte, H.S.S., Parreira, D.F., Ferreira, P.A., Lanza, F.E
Costa, R.V., Casela, C.R., Cota, L.V., 2010. Influence of leaf position that corre-
spond to whole plant severity and diagrammatic scale for white spot of corn.
Crop Prot. 29, 1015e1020.
Cabral, R.N., Marouelli, W.A., Lage, D.A.C., Cafe
-Filho, A.C., 2013. Septoria leaf spot in
organic tomatoes under diverse irrigation systems and water management
strategies. Hort. Bras. 31, 392e400.
Correa, F.M., Bueno Filho, J.S.S., Carmo, M.G.F., 2009. Comparison of three dia-
grammatic keys for the quantification of late blight in tomato leaves. Plant
Pathol. 58, 1128e1133.
Correll, J.C., Gordon, T.R., Elliott, V.J., 1988. Powdery mildew of tomato: the effect of
planting date and triadimefon on disease onset, progress, incidence and
severity. Phytopathology 78, 512e519.
Diver, S., Kuepper, G., Born, H., 1999. Organic Tomato Production. Horticultural
Production Guide. ATTRA, USDA, Fayetteville.
Drinkwater, L.E., Letourneau, D.K., Workneh, F., Van Bruggen, A.H.C., Shennan, C.,
1995. Fundamental differences between conventional and organic tomato
agroecosystems in California. Ecol. Appl. 5, 1098e1112.
Duarte, H.S.S., Zambolim, L., Capucho, A.S., Junior, A.F.N., Rosado, A.W.C.,
Cardoso, C.R., Paul, P.A., Mizubuti, E.S.G., 2013. Development and validation of a
set of standard area diagrams to estimate severity of potato early blight. Eur. J.
Plant Pathol. 137, 249e257.
Gonza
lez-Domínguez, E., Martins, R.B., Del Ponte, E.M., Michereff, S.J., García-
Jimenez, J., Armengol, J., 2014. Development and validation of a standard area
diagram set to aid assessment of severity of loquat scab on fruit. Eur. J. Plant
Pathol. 139, 1e10.
Guzman-Plazola, R.A., Davis, R.M., Marois, J.J., 2003. Effects of relative humidity and
high temperature on spore germination and development of tomato powdery
mildew (Leveillula taurica). Crop Prot. 22, 1157e1168.
Guzm
an-Plazola, R.A., Fajardo-Franco, M.L., Coffey, M.D., 2011. Control of tomato pow-
dery mildew (Leveillula taurica) in the Comarca Lagunera, Coahuila State, Mexico,
supported by the spray forecast model tomato PM. Crop Prot. 30, 1006e1014.
Jones, J.B., Jones, J.P., Stall, R.E., Zitter, T.A., 1991. Compendium of Tomato Diseases.
APS Press, St Paul.
Jones, W.B., Thomson, S.V., 1987. Source of inoculum, yield and quality of tomato as
affected by Leveillula taurica. Plant Dis. 71, 266e268.
Kasselaki, A.M., Shaw, M.W., Malathrakis, N.E., Haralambous, J., 2006. Control of
Leveillula taurica in tomato by Acremonium alternatum is by induction of
resistance not hyperparasitism. Eur. J. Plant Pathol. 115, 263e267.
Kiss, L., Cook, R.T.A., Saenz, G.S., Cunnington, J.H., Takamatsu, S., Pascoe, I.,
Bardin, M., Nicot, P.C., Sato, Y., Rossman, A.Y., 2001. Identification of two pow-
dery mildew, Oidium neolycopersici sp. nov. and Oidium lycopersici, infecting
tomato in different parts of the world. Mycol. Res. 105, 684e697.
Konstantinidou-Doltsinis, S., Markellou, E., Kasselaki, A.M., Fanouraki, M.N.,
Koumaki, C.M., Schmitt, A., Liopa-Tsakalidis, A., Malathrakis, N.E., 2006. Efficacy
of Milsana, a formulated plant extract from Reynoutria sachalinensis, against
powdery mildew of tomato (Leveillula taurica). BioControl 51, 375e392.
Lin, L.I., 1989. Aconcordance correlation coefficient to evaluate reproducibility.
Biometrics 45, 255e268.
Madden, L.V., Hughes, G., Van den Bosch, F., 2007. The Study of Plant Disease Epi-
demics. APS Press, St. Paul.
Nita, M., Ellis, M.A., Madden, L.V., 2003. Reliability and accuracy of visual estimation
of Phomopsis leaf blight of strawberry. Phytopathology 93, 995e1005.
Nutter Jr., F.W., Gleason, M.L., Jenco, J.H., Christians, N.C., 1993. Assessing the ac-
curacy, intra-rater repeatability, and inter-rater reliability of disease assessment
systems. Phytopathology 83, 806e812.
Nutter Jr., F.W., Schultz, P.M., 1995. Improving the accuracy and precision of disease
assessments: selection of methods and use of computer-aided training pro-
grams. Can. J. Plant Pathol. 17, 174e184.
Nutter, F.W., Esker, P.D., 2006. The role of psychophysics in phytopathology. Eur. J.
Plant Pathol. 114, 199e213.
Nutter, F.W., Worawitlikit, O., 1989. Disease.Pro: a computer program for evaluating
and improving a person ability to assess disease proportion. Phytopathology 79,
111e135.
34 D.A. Costa Lage et al. / Crop Protection 67 (2015) 26e34
Nutter, F.W., Teng, P.S., Shokes, F.M., 1991. Disease assessment term and concepts.
Plant Dis. 75, 1187e1188.
Palti, J., 1988. The Leveillula mildews. Bot. Rev. 54, 423e535.
Pedroso, C., Lage, D.A.C., Henz, G.P., Cafe
-Filho, A.C., 2011. Development and vali-
dation of a diagrammatic scale for estimation of anthracnose on sweet pepper
fruits for epidemiological studies. J. Plant Pathol. 93, 219e225.
Reis, A., Boiteux, L.S., Paz-Lima, M.L., Silva, P.P., Lopes, C.A., 2005. New hosts of
Oidiopsis haplophylli in the Solanaceae family in Brazil. Fitopatol. Bras. 30,
195e198.
Reis, A., Boiteux, L.S., Paz-Lima, M.L., 2007. Powdery mildew of ornamental species
caused by Oidiopsis haplophylli in Brazil. Summa Phytopathol. 33, 405e408.
Rios, J.A., Debona, D., Duarte, H.S.S., Rodrigues, F.A., 2013. Development and vali-
dation of a standard area diagram set to assess blast severity on wheat leaves.
Eur. J. Plant Pathol. 136, 603e611.
Rotem, J., Palti, J., 1969. Irrigation and plant diseases. Annu. Rev. Phytopathol. 7,
267e288.
Shoukri, M.M., Pause, C.A., 1999. Statistical Methods for Health Science, second ed.
CRC Press, Boca Raton, FL.
Sutton, T.B., Jones, A.L., 1979. Analysis of factors affecting dispersal of Podosphaera
leucotricha conidia. Phytopathology 69, 380e383.
Vale, F.X.R., Fernandes-Filho, E.I., Liberato, J.R., 2003. A software for plant disease
severity assessment. In: Annals of XVIII International Congress of Plant Pa-
thology. New Zealand, Christchurch, p. 105 (Abstract).
Yadav, N.V., Vos, S.M., Bock, C.H., Wood, B.W., 2013. Development and validation of
standard area diagrams to aid assessment of pecan scab symptoms on fruit.
Plant Pathol. 62, 325e335.
Yi, Q., Wang, P.P., He, Y., 2008. Reliability analysis for continuous measurements:
equivalence test for agreement. Stat. Med. 27, 2816e2825.