Molecular Phylogenetics and Evolution
Vol. 17, No. 3, December, pp. 401– 406, 2000
doi:10.1006/mpev.2000.0845, available online at http://www.idealibrary.com on
Noise and Incongruence: Interpreting Results of the Incongruence
Length Difference Test
Konrad Dolphin,* Robert Belshaw,* C. David L. Orme,* and Donald L. J. Quicke* ,†
*Department of Biology, Imperial College at Silwood Park, Ascot, Berkshire SL5 7PY, United Kingdom; and †Department of
Entomology, The Natural History Museum, London SW7 5BD, United Kingdom
Received December 15, 1999; revised August 4, 2000; published online November 3, 2000
Incongruence between data sets is an important
concept in molecular phylogenetics and is commonly
measured by the incongruence length difference (ILD)
test (J. S. Farris et al., Cladistics 10, 315–319). The ILD
test has been used to infer specific evolutionary events
and to determine whether to combine data sets for
phylogenetic analysis. However, the interpretation in
the literature of the test’s results varies because au-
thors have conflicting expectations of the effect that
noise will have. Using simulations we demonstrate
that noise can by itself generate highly significant re-
sults in the ILD test and demonstrate why this is the
case. To clarify the interpretation of test results, we
suggest an additional procedure in which the result is
compared against a frequency distribution generated
from completely shuffled data. As examples, we apply
this approach to two previous studies that have re-
ported incongruence. © 2000 Academic Press
INTRODUCTION
In a phylogenetic context, congruence is the extent to
which estimates of phylogeny based on different data
sets are in mutual agreement (Page and Holmes, 1998;
Swofford et al., 1996). The concept plays an important
role in phylogenetic research, especially with molecu-
lar data sets in which several biological processes, e.g.,
lineage sorting, gene duplication, and introgression
(Page and Holmes, 1998), can lead to the estimation of
different phylogenies of the same taxa from different
genes.
Congruence may take the form of taxonomic congru-
ence, in which tree topologies are compared; character
congruence, in which the data sets themselves are di-
rectly compared in some way; or a combination of these
two forms. One of the most intuitively appealing and
widespread of the character congruence measures is
the incongruence length difference (ILD) test of Farris
et al. (1994). (Note that this paper is cited in the liter-
ature as both Farris et al. (1994) and Farris et al.
(1995): this is due to the late publication of the 1994
401
volume of Cladistics, which, although published in
1995, bears the date 1994). As we describe below in
more detail, the ILD test is commonly used to test
hypotheses of evolutionary events, and it is used when
considering whether to analyze multiple data sets sep-
arately or simultaneously. We believe that both of
these uses are compromised by the current lack of a
clear understanding of the relationship between the
ILD test and noise, defined here as random data (after
Wenzel and Siddall, 1999) which by definition cannot
reveal any features of shared phyletic history.
The ILD Test
Mickevich and Farris (1981) introduced the ILD
measure and it was developed (and also named) by
Farris et al. (1994).
“For matrices X and Y the incongruence length dif-
ference D xy is given by:
D xy ⫽ L (x⫹y) ⫺ 共L x ⫹ L y兲
L x, L y and L (x⫹y) denote the lengths of most parsimoni-
ous trees calculated for each matrix separately and for
the combined matrix, that including all the charac-
ters.”
To obtain a measure of the significance of this length
difference, a null length distribution is required, and in
practice the test works as follows. (1) Calculate the
sum of most-parsimonious tree (MPT) lengths from the
two matrices (which we will refer to as partitions in the
context of an ILD test). (2) Create replicate partitions
by pooling all characters and randomly allocating them
to two partitions equal in size to that of the originals.
(3) Calculate the sum of the MPT lengths from each of
these replicate partitions to form a tree-length distri-
bution. (4) Calculate the probability that the sum of
lengths from the original partitions (step 1) lies within
this distribution: a low probability implies incongru-
ence.
The test is implemented in this way in software
applications such as arn (Farris, 1991) and PAUP*
(Swofford, 1998); in the latter, the test is known as the
“partition homogeneity” test.
1055-7903/00 $35.00
Copyright © 2000 by Academic Press
All rights of reproduction in any form reserved.
402 DOLPHIN ET AL.
The principle of the test is that if characters are more
congruent within than between partitions, then ho-
moplasy, and therefore tree lengths, will be minimized
when the characters are maintained in their original
partitions and the ILD measure will be large.
Applications of the ILD Test
Many studies have used the ILD test to infer the
probability of evolutionary events using incongruence
between data sets. Recent examples include tests for
hybridization (Doyle et al., 1999), horizontal gene
transfer (Cho et al., 1998; Lecointre et al., 1998), re-
combination (Geiser et al., 1998), large-scale conver-
gence (Muona, 1995; Quicke and Belshaw, 1999), and
hybrid origins of parthenogenesis (Normark and Lan-
teri, 1998). For these purposes, the generation of sig-
nificant results in the test by noise would be positively
misleading.
Another common application of the test is during the
“conditional combination” or “prior agreement” ap-
proach to phylogeny reconstruction (Bull et al., 1993),
in which the analysis of two data sets is carried out
separately if they are found to be significantly incon-
gruent (by some measure) and simultaneously if they
are not (see, for example, Johnson and Sorensen, 1998;
Vidal and Lecointre, 1998; Carbone et al., 1999; Hoot et
al., 1999; Spangler and Olmstead, 1999). Leaving aside
the debate over separate versus combined analysis
(e.g., Chippindale and Wiens, 1994; Nixon and Carpen-
ter, 1996), there is currently an unresolved issue con-
cerning the precise effect that combining noisy and less
noisy data sets has on phylogeny reconstruction (Wen-
zel and Siddall, 1999; Källersjö et al., 1999). This
means that a test which is used to decide whether or
not to combine data sets should not confound both
noise and incongruent signal into a single result.
The Influence of Noise on the ILD Test
There is no consensus in the literature on how noise
will affect the ILD test. The test was published with
the stated expectation that noise would not affect the
result (Farris et al., 1994), and this expectation has
entered standard texts on phylogenetic methodology
(Kitching et al., 1998). Nonetheless, authors differ in
their opinion as to the relationship between noise or
homoplasy and incongruence. For example, Swofford
(1991), quoted in Farris et al. (1994), states that the
ILD test is not influenced by different levels of ho-
moplasy within the matrices but that it should be;
Vidal and Lecointre (1998) agree that it is not influ-
enced by noise, but they follow Farris et al. (1994) in
believing that it should not be; Graham et al. (1998)
come to the conclusion that it is in fact influenced by
noise and offer advice on how to solve this problem (by
excluding poorly supported branches). Other authors
make the observation during their work that some
significant results in the test may be due to a noisy or
small data set, or one which lacks the ability to fully
resolve trees, without addressing the issue further (for
example Cannatella et al., 1998; Piercey-Normore et
al., 1998). This is clearly an unsatisfactory position.
We consider that the problem needs urgent atten-
tion, given the inherently noisy nature of molecular
data, combined with the trend toward multiple gene
analyses and the availability of incongruence tests in
standard phylogeny reconstruction software. We be-
lieve that our study is the first to formally investigate
the relationship between noise and incongruence in the
ILD test, although several previous studies have
touched upon this. Some studies (e.g., Graham et al.
1998) have shown that incongruence can exist between
real and shuffled partitions; others (Cunningham,
1997; Messenger and McGuire, 1998; Stanger-Hall and
Cunningham, 1998) show that the reduction or exclu-
sion of noise from matrices can reduce levels of incon-
gruence. Quicke and Belshaw (1999) showed that the
addition of noise to a small, simulated matrix reduced
incongruence with a simulated matrix of incongruent
characters. We therefore investigated, using simula-
tions, how and why noise can affect the ILD test.
MATERIALS AND METHODS
We created two data matrices, called Comb and
Bush, each of 16 taxa with 13 characters forming a
perfectly pectinate or symmetrical cladogram, respec-
tively. No characters were homoplastic and each node
was supported by one character (Fig. 1). All analyses
were conducted separately for each matrix to reveal
any effects of topology on the noise–incongruence rela-
tionship.
Effect of Noise on ILD Test Results
We assessed the incongruence between each struc-
tured matrix and a range of data partitions containing
varied amounts of noise. ILD tests were conducted
between each matrix and a duplicate of that matrix
which had been modified by shuffling character states,
within an increasing number (from 1 to 13) of ran-
domly selected characters, using a macro in Microsoft
Excel. Examples of shuffled matrices are presented in
Fig. 1. The analyses were carried out on 30 shuffled
replicate partitions for each level of noise. Options on
PAUP* (version 64) were set to 100 replicates of heu-
ristic search using simple addition, MULPARS, and an
unlimited maxtree setting.
Exploring the Response of the ILD Test to Noise
To interpret the results of the previous section, we
proceeded to investigate the relationship between the
amount of noise in a matrix and the length of the
most-parsimonious tree (MPT) which it would produce.
Using replicate matrices with varying amounts of noise
generated as above, we searched for MPTs using
 NOISE AND INCONGRUENCE 403

FIG. 1. The data matrices used in this study, their respective cladograms, and examples showing replicates with four characters shuffled
(boldface).

PAUP* branch and bound option with an unlimited

maxtree setting. The analyses were carried out on 30
shuffled replicates for each level of noise.

RESULTS

Effect of Noise on ILD Test Results

Both data matrices show increasing incongruence
with their duplicate as the proportion of shuffled char-
acters in the duplicated set increases (Fig. 2). Signifi-
cant results appear when 50 – 60% of the characters
represent noise. Incongruence can therefore exist be-
tween one data matrix and another in which nearly
half the characters are perfectly congruent with the
first matrix and there is no additional systematic sig-
nal above that expected by chance alone. FIG. 2. Percentage of ILD tests showing significant incongruence
at the 5% level when comparing a homoplasy-free matrix and a
We stress that this result cannot be predicted simply duplicate with an increasing number of characters shuffled. Results
from the test’s design. We would expect, as is suggested from both the bush matrix (triangles, solid line) and the comb matrix
by Farris et al. (1994), that the test should act as the (circles, dotted line) are shown.
404 DOLPHIN ET AL.
FIG. 3. (a) Most-parsimonious tree length grows in a nonlinear
fashion with an increasing proportion of noise in each matrix. Re-
sults from both the bush matrix (triangles) and the comb matrix
(circles) are shown. Standard error bars are too small to be visible.
(b) The effect of this curvilinear relationship on the ILD test. Points
A and B represent two equally sized matrices; A is relatively noise-
free and produces a shorter tree than the noisier B. When the
characters (and noise) are repartitioned during the ILD test, the
length of the average replicate is closer to B than to A.
equivalent of the Mann–Whitney U test and should
assess a difference in mean (⫽ signal) without being
affected by a difference in variance (⫽ noise) in the two
samples. Our suggestion as to the cause of this ILD
significance is described below.
Exploring the Response of the ILD Test to Noise
Figure 3a illustrates how MPT length grows with the
increasing amount of noise in a matrix. This relation-
ship is nonlinear: a linear relationship would hold if
each additional shuffled character contributed the
same number of extra steps to the tree length. In fact,
as we progressively increase the number of noisy char-
acters, the average length of each shuffled character on
the MPT decreases and the average length of each
nonshuffled character increases until their average
lengths become identical at extremely high noise lev-
els. This means that at the noisy end of the spectrum it
is less costly in terms of tree length to shuffle each
additional character, and the resulting length to noise
relationship is curvilinear.
As a consequence of this relationship, matrices with
intermediate levels of noise produce longer trees than
predicted by a linear model. Figure 3b shows the effects
that this has on the ILD test. Matrix A is 20% noise
and gives a tree length of 22 steps, matrix B is of equal
size, but contains 80% noise and gives a tree length of
33 steps: a total initial partition tree length of 55 steps.
When the characters are repartitioned, each matrix
will contain on average 50% noise and give a replicate
sum of tree lengths totalling (29 ⫻ 2) ⫽ 58 steps, which
is 3 steps longer than the original. Although the num-
bers are arbitrary in this example, it is generally true
that replicate partitions with an intermediate amount
of noise will produce a longer sum of tree lengths than
an original partition comprising one noisy and one less
noisy matrix. When this difference reaches a certain
level, ILD test results will be significant even in the
absence of systematic incongruence.
DISCUSSION
Our findings that significance in the ILD test can be
increased by a difference in levels of noise in two par-
titions have several implications for phylogenetic re-
search.
(a) Where the ILD test is used to test for particular
events during molecular evolution (recombination, in-
trogression, etc.) by assessing the extent of incongru-
ence between two sequences, the unknown amount of
significance generated by different levels of homoplasy
within the data sets will render the interpretation of P
values generated by the test ambiguous.
(b) The ILD test is increasingly being used to de-
termine whether combined or separate analysis of par-
titions is favorable. The results of this study show that
the ILD test could lead to a separate analysis of two
matrices representing a similar or identical underlying
topology but whose characters had evolved at different
rates and thus displayed different amounts of noise.
Whether this consequence is advantageous is open to
debate (but see Wenzel and Siddall, 1999 for a discus-
sion of combining noisy and less noisy data sets). None-
theless, with currently unquantifiable contributions to
the significance of the test coming from both incongru-
ent signal and noise, the precise meaning of the ILD
test result will be ambiguous.
(c) On a related topic, some of the recent debate
concerning the utility or otherwise of 3rd codon posi-
tions in phylogeny reconstruction has been based on
the interpretation of incongruence inferred from the
ILD test. The opposing views regarding the treatment
of 3rd codon characters are summarized by Vidal and
Lecointre (1998). Some researchers believe that 3rd
codon characters can create actual conflict deep within
the tree and should thus be down-weighted to mini-
mize incongruence. Others believe that 3rd codon po-
sitions should be included in analyses since they are
informative at subterminal nodes and simply noisy at
deeper levels (and so are not destructive of the signal
from more conservative characters). A discussion of the
positive consequences of including these characters
can be found in work by Källersjö et al. (1999). Vidal
and Lecointre (1998) demonstrate that there is more
incongruence between codon positions than between
separate genes using the ILD test and cite this as
evidence that 3rd codon positions are sometimes in
significant conflict with other positions rather than
being simply noisy. Thus, they refute the view that 3rd
codon characters should always be included in analy-
ses because of their interpretation of the ILD test re-
sults.
 NOISE AND INCONGRUENCE
We propose that an alternative null model for the
ILD test, which makes allowance for the major effect of
noise demonstrated in our study, would be useful in
some cases. To detect incongruence that cannot be
attributed to noise alone we recommend the following
procedure: (1) perform an ILD test on the two original
data partitions (A and B); (2) shuffle the character
states within the characters of one matrix (A) repeat-
edly to generate a series of pure noise partitions; (3)
carry out ILD tests between these shuffled replicates
and the original matrix B; (4) repeat the procedure
comparing shuffled replicates of B against unshuffled
A; (5) only if the original level of conflict in step 1 is
significantly higher than the levels observed in both
steps 3 and 4 can we say that the partitions are signif-
icantly incongruent.
Shuffling character states within characters retains
features of a matrix such as frequency and between-
character distribution of 0’s and 1’s while replacing
signal with noise. Any difference in the ILD test results
obtained before and after shuffling one matrix in this
way indicates features of agreement or incongruence
that cannot be attributed to noise in the initial parti-
tions (although other effects, such as base composition
bias, cannot be discounted).
Two examples of this approach are the following. (1)
We further analyzed the mitochondrial data examined
in Cunningham (1997) in which partitions represent
1st, 2nd, and 3rd codon positions with the 3rd codon
positions saturated by multiple substitutions (a good
example of noise). The initial ILD test using branch-
and-bound option finds the 2nd and 3rd codon parti-
tions to be highly incongruent (P ⬍ 0.001)—a striking
result, given the putatively identical phylogenetic his-
tory of the two partitions. Shuffling the 3rd codon char-
acter states tends to reduce the level of incongruence,
with the mean difference in tree lengths between orig-
inal and replicate partitions being 3.18 S.D. (n ⫽ 30)
compared to 6.93 S.D. when the unshuffled data are
analyzed. However, the value of 6.93 S.D. lies within
the 95% confidence limit calculated for the 30 shuffled
partitions. The importance of distinguishing between
noise and signal in the case of codon position charac-
ters has been mentioned above, and this example high-
lights the need for an unambiguous incongruence mea-
sure. (2) In their extensive study of leptodactylid frogs,
Cannatella et al. (1998) examine phylogenetic matrices
whose characters represent a range of morphological,
behavioral, and molecular sources. Multiple pairwise
ILD tests between the partitions revealed that the only
instance of significant incongruence was when a ma-
trix comprising advertisement call characters (which
they named CALLS) was compared with a morpholog-
ical matrix.
We created a series of shuffled replicates of the
CALLS matrix and performed ILD tests comparing
these replicates with the original morphological matrix
405
(branch-and-bound search, 10,000 replicates). In 10 of
15 instances, the significance of the shuffled replicate
ILD test was greater than that of the original partition.
Thus, we agree with the authors’ hypothesis that the
incongruence is due to the small and noisy nature of
the CALLS matrix.
ACKNOWLEDGMENTS
David Swofford kindly allowed us to publish results obtained from
a test version of his program PAUP*. K.D. was supported by a
BBSRC Grant. R.B. and C.D.L.O. were supported by NERC Grants.
REFERENCES
Bull, J. J., Huelsenbeck, J. P., Cunningham, C. W., Swofford, D. L.,
and Waddell, P. J. (1993). Partitioning and combining data in
phylogenetic analysis. Syst. Biol. 42: 384 –397.
Cannatella, D. C., Hillis, D. M., Chippindale, P. T., Weigt, L., Rand,
A. S., and Ryan, M. J. (1998). Phylogeny of frogs of the Physalae-
mus pustulosus species group, with an examination of data incon-
gruence. Syst. Biol. 47: 311–335.
Carbone, I., Anderson, J. B., and Kohn, L. M. (1999). Patterns of
descent in clonal lineages and their multilocus fingerprints are
resolved with combined gene genealogies. Evolution 16: 354 –362.
Chippindale, P. T., and Wiens, J. J. (1994). Weighting, partitioning,
and combining characters in phylogenetic analysis. Syst. Biol. 43:
278 –287.
Cho, Y., Qiu, Y. L., Kuhlman, P., and Palmer, J. D. (1998). Explosive
invasion of plant mitochondria by a group I intron. Proc. Natl.
Acad. Sci. USA 95: 14244 –14249.
Cunningham, C. W. (1997) Is congruence between data partitions a
reliable predictor of phylogenetic accuracy? Empirically testing an
iterative procedure for choosing among phylogenetic methods.
Syst. Biol. 46: 464 – 478.
Doyle, J. J., Doyle, J. L., and Brown, A. H. D. (1999). Incongruence in
the diploid B-genome species complex of Glycine (Leguminosae)
revisited: Histone H3-D alleles versus chloroplast haplotypes. Mol.
Biol. Evol. 16: 354 –362.
Farris, J. S. (1991). “Arn”. Program and documentation. Molekular-
systematiska laboratoriet, Naturhistoriska riksmuseet, Box 50007
S 104 –105, Stockholm, Sweden.
Farris, J. S., Källersjö, M., Kluge, A. G., and Bult, C. (1994). Testing
significance of incongruence. Cladistics 10: 315–319.
Geiser, D. M., Pitt, J. I., and Taylor, J. W. (1998). Cryptic speciation
and recombination in the aflatoxin-producing fungus Aspergillus
flavus. Proc. Natl. Acad. Sci. USA 95: 388 –393.
Graham, S. W., Kohn, J. R., Morton, B. R., Eckenwalder, J. E., and
Barrett, S. C. H. (1998). Phylogenetic congruence and discordance
among one morphological and three molecular data sets from
Pontederiaceae. Syst. Biol. 47: 545–567.
Hoot, S. B., Magallon, S., and Crane, P. R. (1999). Phylogeny of basal
eudicots based on three molecular data sets: atpB, rbcL and 18S
nuclear ribosomal DNA sequences. Ann. Missouri Bot. Garden 86:
1–32.
Johnson, K. P., and Sorensen, M. D. (1998). Comparing molecular
evolution in two mitochondrial protein coding genes (cytochrome b
and ND2) in the dabbling ducks (Tribe: Anatini). Mol. Phylogenet.
Evol. 10: 82–94.
Källersjö, M., Albert, V. A., and Farris, J. S. (1999). Homoplasy
increases phylogenetic structure. Cladistics 15: 91–93.
Kitching, I. J., Forey, P. L., Humphries, C. J., and Williams, D. M.
406 DOLPHIN ET AL.
(1998). “Cladistics: The Theory and Practice of Parsimony Analy-
sis,” 2nd ed., Oxford Univ. Press, Oxford.
Lecointre, G., Rachdi, L., Darlu, P., and Denamur, E. (1998). Esch-
erichia coli molecular phylogeny using the incongruence length
difference test. Mol. Biol. Evol. 15: 1685–1695.
Messenger, S. L., and McGuire, J. A. (1998). Morphology, molecules,
and the phylogenetics of cetaceans. Syst. Biol. 47: 90 –124.
Mickevich, M. F., and Farris, J. S. (1981). The implications of con-
gruence in Menidia. Syst. Zool. 30: 351–370.
Muona, J. (1995). The phylogeny of Elateroidea (Coleoptera), or
which tree is best today? Cladistics 11: 317–341.
Nixon, K. C., and Carpenter, J. M. (1996). On simultaneous analysis.
Cladistics 12: 221–241.
Normark, B. B., and Lanteri, A. A. (1998). Incongruence between
morphological and mitochondrial DNA characters suggests hybrid
origins of parthenogenetic weevil lineages (genus Aramigus). Syst.
Biol. 47: 475– 494.
Page, R. D. M., and Holmes, E. C. (1998). “Molecular Evolution: A
Phylogenetic Approach,” Blackwell Sci., Oxford.
Piercey-Normore, M. D., Egger, K. N., and Bérubé, J. A. (1998).
Molecular phylogeny and evolutionary divergence of North Amer-
ican species of Armillaria. Mol. Phylogenet. Evol. 10: 49 – 66.
Quicke, D.L.J., and Belshaw, R. (1999). Incongruence between mor-
phological data sets: An example from the evolution of endopara-
sitism among parasitic wasps (Hymenoptera: Braconidae). Syst.
Biol. 48: 436 – 454.
Spangler, R. E., and Olmstead, R. G. (1999). Phylogenetic analysis of
Bignoniaceae based on the cpDNA gene sequences rbcL and ndhF.
Ann. Missouri Bot. Garden 86: 33– 46.
Stanger-Hall, I. K., and C. W. Cunningham. (1998). Support for a
monophyletic Lemuriformes: Overcoming incongruence between
data partitions. Mol. Biol. Evol. 15: 1572–1577.
Swofford, D. L. (1991). When are phylogeny estimates from molecu-
lar and morphological data incongruent? In “Phylogenetic Analysis
of DNA Sequences” (M. Miyamoto and J. Cracraft, Eds.), pp. 295–
333. Oxford Univ. Press, Oxford.
Swofford, D. L. (1998). PAUP*. Phylogenetic Analysis Using Parsi-
mony (*and other methods). Version 4. Sinauer, Sunderland, MA.
Swofford, D. L., Olsen, G. J., Waddell, P. J., and Hillis, D. M. (1996).
Phylogenetic inference. In “Molecular Systematics” (D. M. Hillis,
C. Moritz, and B. K. Mable, Eds.), pp. 407–514. Sinauer, Sunder-
land, MA.
Vidal, N., and Lecointre, G. (1998). Weighting and congruence: A
case study based on three mitochondrial genes in pitvipers. Mol.
Phylogenet. Evol. 9: 366 –374.
Wenzel, J. W., and Siddall, M. E. (1999). Noise. Cladistics 15: 51– 64.