ORIGINAL ARTICLES

Methodology for Speech Assessment in the Scandcleft Project—

An International Randomized Clinical Trial on Palatal Surgery:
Experiences From a Pilot Study
A. Lohmander, Ph.D., E. Willadsen, Ph.D., C. Persson, Ph.D., G. Henningsson, Ph.D., M. Bowden, B.Sc., B. Hutters, M.A.

Objective: To present the methodology for speech assessment in the

Scandcleft project and discuss issues from a pilot study.
Design: Description of methodology and blinded test for speech assessment.
Speech samples and instructions for data collection and analysis for comparisons
of speech outcomes across five included languages were developed and tested.
Participants and Materials: Randomly selected video recordings of 10 5-year-
old children from each language (n = 50) were included in the project. Speech
material consisted of test consonants in single words, connected speech, and
syllable chains with nasal consonants. Five experienced speech and language
pathologists participated as observers.
Main Outcome Measures: Narrow phonetic transcription of test consonants
translated into cleft speech characteristics, ordinal scale rating of resonance, and
perceived velopharyngeal closure (VPC). A velopharyngeal composite score (VPC-
sum) was extrapolated from raw data. Intra-agreement comparisons were performed.
Results: Range for intra-agreement for consonant analysis was 53% to 89%,
for hypernasality on high vowels in single words the range was 20% to 80%, and
the agreement between the VPC-sum and the overall rating of VPC was 78%.
Conclusions: Pooling data of speakers of different languages in the same trial
and comparing speech outcome across trials seems possible if the assessment of
speech concerns consonants and is confined to speech units that are phonetically
similar across languages. Agreed conventions and rules are important. A composite
variable for perceptual assessment of velopharyngeal function during speech seems
usable; whereas, the method for hypernasality evaluation requires further testing.

KEY WORDS: cleft palate, composite score, cross-linguistic, multicenter

comparison, speech assessment, speech outcome measures

In 1997, cleft palate teams in Scandinavia decided to start

Dr. Lohmander is Professor and Speech and Language Pathologist,
Division of Speech and Language Pathology, Sahlgrenska Academy at
University of Gothenburg, Sweden. Dr. Willadsen is Assistant Professor
and Speech and Language Pathologist, Department of Applied Linguis-
tics, University of Copenhagen, Denmark. Dr. Persson is Speech and
Language Pathologist and Lecturer, Division of Speech and Language
Pathology, Sahlgrenska Academy at University of Gothenburg, Sweden.
Dr. Henningsson is Associate Professor and Speech and Language
Pathologist, Department of Logopedics and Phoniatrics, Karolinska
Institute, Stockholm, Sweden. Dr. Bowden is Lead Specialist Speech and
Language Therapist, Greater Manchester Cleft Lip and Palate Unit,
University Dental Hospital of Manchester, United Kingdom. Dr. Hutters
is Phonetician and Associate Professor, Department of Applied Linguis-
tics, University of Copenhagen, Denmark.
This work was supported, in part, by European Union: EUROCRAN
Collaboration on Craniofacial Anomalies, QLG1-CT-2000-01019 and
Accompanying Measures QLG1-CT-2002-30587.
Submitted September 2008; Accepted November 2008.
Address correspondence to: Prof. Anette Lohmander, Division of
Speech and Language Pathology, Sahlgrenska Academy at University of
Gothenburg, PO Box 452, SE 405 30, Gothenburg, Sweden. E-mail
Anette.Lohmander@neuro.gu.se and Anette.Lohmander@ki.se.
DOI: 10.1597/08-039.1
a multicenter study of primary cleft palate repair in
unilateral cleft lip and palate (UCLP), the Scandcleft
project. Shortly afterwards, two British centers joined the
project. At that time, there was no consensus regarding the
surgical protocol for primary palatal repair, and a few years
later, this was demonstrated by a survey of clinical services
for cleft lip and palate in Europe showing that 201 clinical
teams used 194 different protocols for primary surgical
repair of UCLP (Shaw et al., 2000). Shortage of evidence
regarding outcomes is one reason for the lack of consensus,
which concerning speech is due to methodological flaws
and lack of standardized procedures for assessment of
speech outcome (Lohmander and Olsson, 2004).
The Scandcleft project was designed as a randomized
controlled trial to test treatment outcome at age 5 years as a
function of surgical intervention. It is conducted as a
parallel series of three randomized control trials of primary
surgery (Table 1). The trial team comprises different
categories of researchers that have been meeting as a group
since 1997 to develop this study. The participating centers

347
348 Cleft Palate–Craniofacial Journal, July 2009, Vol. 46 No. 4
TABLE 1 Study Design of the Scandcleft Project
Trial No. Participating Center
1 Copenhagen/Aarhus*, Denmark; Gothenburg,
Sweden
2 Helsinki, Finland; Stockholm/Linköping*,
Sweden
3 Oslo/Bergen*, Norway; Belfast and
Manchester,{ U.K.
Common To be carried out by all centers
* 5 same surgical team for patients followed by teams in two cities in one country.
{ 5 different surgical teams in two cities.
are Aarhus and Copenhagen in Denmark, Oslo and Bergen
in Norway, Gothenburg, Linköping and Stockholm in
Sweden, Helsinki in Finland, and Belfast and Manchester
in the U.K. Surgeons from each of the centers have met for
calibration purposes as well as to develop protocols, as
have a group of orthodontists. In addition, protocols for
ear and hearing status and questionnaires on parents’
opinion of treatment and care have been developed by
professionals in the specialist areas.
The general principles of perceptual speech assessment
are data sampling, recording, analysis, and interpretation
(Grunwell et al., 1993). The performance in these areas was
recently reviewed by Sell (2005), which is recommended for
a comprehensive survey. Kuehn and Moller (2000)
suggested that standardized description of valid speech
variables, methods, and procedures for speech assessment
with repeatable and reliable judgments be required, i.e.,
minimal standards for reporting results of surgery on
patients with cleft lip and palate that were proposed 20
years ago (Dalston et al., 1988). Procedures for speech
assessment do, however, continue to vary considerably, and
the validity and reliability of the results can still be
questioned (Lohmander and Olsson, 2004; Sell, 2005;
Fujiwara et al., 2006). It is, for example, of great concern
that another ongoing randomized clinical trial of cleft
surgery presented as a design paper 10 years ago by
Williams et al. (1998) does not seem to contain appropriate
procedures for data recording and analysis. Brief informa-
tion on the speech assessment that will be carried out in the
future is given in the article, but it is concerning that no
audio or video recordings seem to be taken and that
reliability measures were only part of a pilot study to be
carried out prior to the actual speech assessment.
Standardized procedures for data collection (material
and recording) needed to be carefully considered in the
Scandcleft project, and one of the big challenges was to
assess speech in a multilingual patient group. Fortunately,
a proposal on speech assessment with special reference to
minimum standards for intercenter comparison was pre-
sented at the International Congress on Cleft Palate and
Related Craniofacial Anomalies in Singapore (Hennings-
son and Hutters, 1997). The proposal was the start of
further development of the methodology for speech
assessment in the Scandcleft project. The speech and
language trialists (known as the Scandcleft Speech Group)
Language Surgical Procedure
Danish, Swedish lip and soft palate closure at 3–4 months; hard palate closure at 36 months
Finnish, Swedish lip closure at 3–4 months; hard and soft palate closure at 12 months
Norwegian, English lip closure at 3–4 months combined with a single layer closure of the hard
palate using a vomer flap; soft palate closure at 12 months
lip and soft palate closure at 3–4 months; hard palate closure at 12 months
met annually until 2003 to devise and agree on this first
part of the methodology. The method for analysis is still to
be determined and has recently been tested in a pilot study.
Thus, the purpose of this paper is to present and discuss the
agreed part of the methodology in the Scandcleft project,
i.e., the speech material and recording procedures. Fur-
thermore, the purpose was to present and discuss the results
of the pilot study, which will be used for adjustments and
further development of the final method for analysis. The
analysis will be carried out in approximately 3 years, when
all participating patients have had speech recordings taken
at age 5 years.
In the present paper, the focus of speech assessment is on
methodology for cross-linguistic studies. Two recent
publications and a Web site describe the specific prerequi-
sites for cross-linguistic studies and can be referred to for a
detailed description (Hutters and Henningsson, 2004;
Henningsson et al., 2008; www.eurocran.org). A summary
of the conditions is outlined below, followed by general
details on recording and analysis. The methodology for
speech assessment in the Scandcleft project is then
described, and finally the pilot assessment is presented
and discussed.
Procedures for Perceptual Speech Assessment With Refer-
ence to Cross-Linguistic Studies
Hutters and Henningsson (2004) describe the impact of
language as a background variable in studies of speech
outcome as a function of treatment of cleft palate. The
possible influence on a speech sound by the cleft condition
is determined by the phonetic characteristics of the sound.
Because languages differ regarding phonetic characteristics
of consonants, the possible consequences or impact a cleft
palate may have on speech quality varies from language to
language. Consequently, as with other independent vari-
ables that may influence the dependent variable, in this case
speech outcome, the influence of the language variable
should be eliminated. Speech outcome data in cross-
linguistic studies should be based on speech units that are
phonetically similar across the languages and which occur
in similar contexts. Therefore, the speech sound inventories
of the languages involved in a study should be compared.
All speech sounds may be influenced by the cleft condition,
but only a restricted number of consonants may be similar
 Lohmander et al., SPEECH ASSESSMENT IN THE SANDCLEFT PROJECT 349
across the languages involved. Accordingly, using a
restricted number of phonetically similar speech sounds
enhances the validity of the cross-linguistic outcome data if
the assessment is made primarily on speech sounds that are
highly vulnerable to the cleft condition, i.e., pressure
consonants. Preferably, these consonants should be in
‘‘strong position’’ in a word, i.e., the position where a
speech sound is most distinctly articulated, most easily
recognizable, and minimally influenced by the phonetic
context. The phonetic context of the target sounds should
be similar and not include nasal consonants or consonant
clusters. For assessment of hypernasal resonance, test
words including high vowels should be added for assess-
ment of hyponasal resonance, words with nasal consonants
should be added (Henningsson and Hutters, 1997). Speech
material designed in this way will be shorter than, for
example, an ordinary language-specific articulation test,
which typically includes all consonants in the language.
However, it should be stressed that in studies of speech
outcome following surgical treatment, speech outcome data
based on a restricted number of different sounds are
sufficient as opposed to data intended for clinical purposes,
which need to be more detailed (Hutters and Henningsson,
2004).
The Eurocleft speech project (Brøndsted et al., 1994), the
only known published cross-linguistic study, was used by
Hutters and Henningsson (2004) to illustrate some meth-
odological problems in studies of speech outcome following
treatment in speakers with different language background.
For example, the selection of common consonants across
the included five Germanic languages was based on
phonetic criteria in the broad sense of the word (broad
phonetic transcription). With proportionately old speakers
(10 years of age) with generally no or small speech
problems, these broad phonetic criteria were considered
invalid by Hutters and Henningsson (2004). They stated
that the ‘‘…context of the test consonants did not seem to
be taken into consideration in terms of the full conse-
quences that the phonetic characteristics of a test sound
and realization of a phoneme may vary with segmental and
prosodic context.’’ Also, the Scandcleft speech project,
although not yet published, was used to illustrate the
methodological problems in cross-linguistic studies. The
impact of including a Finno-Ugric language (Finnish) on
the selection of target consonants was discussed. In
addition, considerations and decision by the Scandcleft
Speech Group on the final speech materials to be used with
preschool children were given and are now presented in the
present paper in the paragraph on Methodology in the
Scandcleft project.
Henningsson et al. (2008) reported on a proposal of
universal parameters for reporting speech outcomes con-
taining general considerations and guidelines for speech
sampling. Nothing else being similar, the considerations
and guidelines for speech sampling are the same as those of
the EUROCRAN speech project (www.eurocran.org). This
is a Web site presenting both general principles and
guidelines for speech sampling on various purposes as well
as listening exercises for training. The general consider-
ations and guidelines have also previously been published
by Hutters and Henningsson (2004).
Recording
All speech samples should be collected in a standardized
manner with regard to the setting and recording (Sell,
2005). Audio and/or video recording should be made for
later assessment. Only recordings of speech samples allow
the obtainment of intra- and interreliability measures of
perceptual speech assessment. Good quality recording
equipment should be used to allow assessment of different
speech quality variables.
Analysis
Perceptual Analysis of Consonant Articulation
Currently, speech outcomes are reported as degrees and/
or prevalences of speech errors obtained from rating scales
or from phonetic transcription (for a summary see Sell,
2005). In general, rating scales are considered of limited
value for measuring consonant production, in part due to
poor definitions (Sell, 2005). Based on phonetic transcrip-
tion, typical cleft speech characteristics can be identified
and further grouped into speech summary patterns
typically according to place of articulation: anterior errors,
posterior oral errors, nonoral errors (Harding et al., 1997),
and further into the main categories active and passive
characteristics (Hutters and Brøndsted, 1987; Harding and
Grunwell, 1998). Active characteristics are related to a
reduced ability or inability to produce the phonemes on
correct articulation places and are often called compensa-
tory articulation. These were further divided into the
categories anterior oral, posterior oral, and posterior
nonoral according to the placement of the articulation
error by Harding et al., (1997). Passive characteristics like
nasal air leakage and weak pressure consonants are a direct
result of an inability to make closure between the oral and
nasal cavity, i.e., velopharyngeal impairment or a fistula1
(Henningsson and Isberg, 1990). This is the case in
GOS.SP.ASS/CAPS (Harding et al., 1997; Sell et al.,
1999) as well as in the Swedish Articulation and Nasality
Test (SVANTE) (Lohmander et al., 2005). A system with
clinically relevant categories for interpretation is practical
and enhances communication with the patient, between the
professionals in a team, and with relatives. It is, however,
1
Fistula friction is perceived as a whistling sound; whereas, those
related to VPI are either nasal friction sound, also called nasal air
emission, perceived when air is passing through the nose, or velopharyn-
geal friction, perceived as a snorting sound that is generated in the
velopharynx.
350 Cleft Palate–Craniofacial Journal, July 2009, Vol. 46 No. 4
crucial that the categorizations are made according to the
same rules or definitions. Otherwise, comparisons of
outcomes will be meaningless (Sell, 2005). This is exempli-
fied by Sell (2005) with ‘‘palatal errors’’ that are
summarized as anterior errors in the U.K. system, posterior
errors in the Swedish system, and as compensatory errors in
the U.S. system (Trost, 1981).
Henningsson et al. (2008) also gives definitions and
detailed descriptions of suggested speech parameters for
analysis including five ‘‘universal’’ reporting parameters:
hypernasality, hyponasality, nasal air emission and/or
turbulence, consonant production errors, and voice disor-
der. The consonant production errors comprise eight
subcategories, which are similar to those in different
national systems in use (see above).
Speech outcome is not only influenced by the language of
the speakers’ language background. It is well documented
that description of speech sounds based on auditory
assessment is influenced by the listeners’ language back-
ground (e.g., Coussé et al., 2004). The Danish language has,
for example, no voiced plosives—the distinctive feature
being one of aspiration, and therefore it may be difficult for
a Danish speaker to reliably judge the voicing feature of
plosives produced by, e.g., an English speaking child and
vice versa (Hutters and Henningsson, 2004) (see Appendix
1 for the Danish words such as ‘‘piger’’ and ‘‘puder’’). As a
consequence, in studies including different languages, the
speech data should be assessed by specialists speaking the
same language as the speaker being analyzed in order to
eliminate a possible influence of the language spoken by the
observer. Using the International Phonetic Alphabet (IPA)
(and the extended IPA for disordered speech [ExtIPA]),
narrow transcription will allow the observer to describe the
required level of phonetic detail of the target consonant
(International Phonetic Association, 1999). Although the
consonants are similar across languages, there are differ-
ences as to voicing and aspiration and subtle differences in
place and manner of articulation. This means that the
influence or impact of the cleft condition on the consonants
may differ across languages that are compared. This
potential artifact should be recognized and taken into
account in the final analysis of the speech outcome data.
Detailed phonetic transcription provides the necessary
information for such an analysis, even though limited
reliability has been reported (Shriberg and Lof, 1991; Kent,
1996). However, training is supposed to increase the level of
agreement (Keunig et al., 1999; Gooch et al., 2001; John et
al., 2006) and should preferably focus on phonetic features
considered particularly problematic (Howard and Hesel-
wood, 2002).
Perceptual Analysis of Nasal Resonance
A description of the resonance characteristics is usually
included in the assessment of speakers with cleft palate
(Kuehn and Moller, 2000). The perception of hypernasality
varies as a function of other aspects of speech. It is more
severe on high vowels than on low vowels, other things
being equal (Spriestersbach and Powers, 1959; Kuehn and
Moon, 1998), and varies according to phonetic context
(Lintz and Sherman, 1961). Therefore, the speech material
should include high vowels in similar phonetic context.
Although the decision as to whether a speaker is hypernasal
is the subjective decision of an observer, perceptual
judgments by observers are still not presented with
convincing reliability. This could be attributed to variable
internal standards acquired by different individuals, i.e., all
observer experiences are thought to be stored in the
memory and are believed to form the internal standards
(Kreiman et al., 1993; Keuning et al., 1999). These authors
have also found that the internal standards for observers
could be unstable over time, independent of the experience
level of the observer. Moreover, it has been indicated that
observers with clinical experience of resonance disorders
have higher interobserver agreement on nasality ratings
than clinically inexperienced but academically trained
observers (Lewis et al., 2003). It is easier to agree on a
normal outcome of a variable than on a pathological one
(Kreiman et al., 1993), and problems other than hyperna-
sality may interfere with the judgment and even mask the
impression of hypernasality (Fletcher and Bishop, 1970;
Kent, 1996). Today, a variety of psychologic scaling
procedures are being used. Kuehn and Moller (2000) stated
that both descriptive category judgments and rating of
severity will continue to be useful in describing changes in
resonance after surgical or behavioral intervention. Con-
tinuing effort to increase the level of agreement is therefore
highly needed.
An overall variable for perceptual assessment of velo-
pharyngeal function has sometimes been used for evalua-
tion of velopharyngeal function during speech (e.g.,
McWilliams et al., 1990). Such an overall variable has
been found representative of the observers’ overall impres-
sion of degree of impairment (Dotevall et al., 2002). In that
study, the different variables ‘‘hypernasality,’’ ‘‘audible
nasal escape,’’ ‘‘velopharyngeal friction sounds,’’ and
‘‘weak pressure consonants’’ were closely interrelated. It
was further demonstrated that a sum of the ratings of these
four variables (composite velopharyngeal score) had a high
correlation with the overall perceptual rating of velopha-
ryngeal function. Overall variables or composite scores
have recently been used in addition to assessment of speech
errors related to cleft palate (John et al., 2006; Lohmander
et al., 2006; Havstam et al., 2008).
METHODOLOGY IN THE SCANDCLEFT PROJECT
All children in the Scandcleft project are documented by
a Speech and Language Pathologist/Therapist (SLP/T)
involved in the project at 12 months, 18 months, 3 years,
5 years (main outcome age), and 10 years of age. For the
purpose of this paper both the agreed upon sampling and
 Lohmander et al., SPEECH ASSESSMENT IN THE SANDCLEFT PROJECT 351
recording of data and the suggested procedure for analysis
at 5 years of age are presented (which is similar to the data
and procedures at 3 and 10 years of age) and then discussed
based on the pilot study.
Data Sampling/Speech Material
Since the purpose of the Scandcleft study is to investigate
the influence of surgical procedure on speech outcome in
speakers of five different languages, restricted speech
material of phonetically similar speech sounds for each
language was selected. The five languages included in the
Scandcleft project are English, Swedish, Danish, Finnish,
and Norwegian. The phonetically similar speech sounds
selected were the target pressure consonants and taking
into account the young age of the participants, the
consonants were imbedded in a single word naming test.
These considerations and problems have previously also
been described by Hutters and Henningsson (2004). The
Scandcleft Speech Group had great difficulty in finding
developmentally suitable vocabulary that met the phonetic
requirements for all languages and spent considerable time
identifying all of the words. The words were finally decided
on after an informal test on typically developed children
showing that they had the words in their vocabulary. Some
needed to be elicited in plural because the indefinite article
with nouns of common gender has a final nasal consonant
(‘‘en’’) in three of the languages. A few words were proper
nouns and were elicited through repetition.
Thus, the speech material consists of 33 single words (see
Appendix 1) elicited through color pictures with a
standardized layout created using the computer software
‘‘Boardmaker’’ (Mayer-Johnson Inc., Solana Beach, CA).
The realization of pressure consonants and nasal resonance
on vowels is the focus of the project. There are three
realizations of each target consonant occurring in three
different words. The target consonants are in strong
position, i.e., they occur in the stressed syllable initial
position followed by a vowel. The words should not include
pressure consonants other than the target consonant, and
no nasal consonants and consonant clusters. However, in
practice we had to include a few words with two pressure
consonants in order to obtain a sufficient number of words
that could be named by young children (Appendix 1). A
potential contextual influence on the test consonant from
the other pressure consonant should be considered in the
analysis of the data. Except for Finnish, the speech material
consists of the plosives /p b t d k g/, the labiodental
fricatives /f v/, and /s/. Finnish has only the plosives /p t k/,
and therefore, the Finnish material includes two sets of
words with /p t k/, respectively, giving six words with
plosives as in the other languages. Because Finnish has no
/f/, the Finnish material includes only 30 words. In order to
investigate /s/ in younger children, final /s/ is also included
because this fricative is managed earlier in final than in
initial position. In some words, the test consonant is
followed by a high vowel intended for assessment of
hypernasality. Three words with the nasal consonant /n/ are
included for assessment of hyponasality. The order of the
words is randomized according to a standard procedure for
randomization.
The speech material for nasometry comprises the high
vowel words repeated five times, a nasal syllable string /ma
ma ma ma ma/. Language-specific sentences are also
included in the collection of nasometry data in the
Scandcleft project only for comparison with language-
specific norms. The material and procedure for nasometry
will be described in more detail in a separate article.
Data Collection/Recording
Simultaneous video and audio recordings of 5-year-old
children with unilateral cleft lip and palate are carried out
in the Scandcleft project according to the following
instructions: ‘‘All 5-year assessments should be both
audiotaped and video recorded and carried out by two
clinicians. The examiner should sit opposite the child with
the camera operator just behind so that the camera lens is
directly facing the child. The two microphones should be
placed at a distance of about 40 cm from the edge of the
table in front of the child and on the same side as the
camera. Use a neutral background and focus on the child
so that their head and the shoulders are in the center of the
viewer with some space on either side. The test pictures
should be presented in such a way that the child is able to
look straight ahead into the camera.’’ (Scandcleft Project
Manual, 1997).
The video recordings are being made using a super-VHS
or digital video camera with external microphone of
excellent quality. Simultaneous audio recordings are
performed with a DAT tape recorder (SONY TCD-D8)
or one of comparable quality with a condenser microphone
(AKG C407/B) or one of comparable quality. Thus, two
separate pieces of equipment are used for the recordings.
Two practice pictures are presented to the child at the
very start. All of the target words should be elicited by
naming. If this fails (the child does not recognize the picture
or know the word), then semantic prompting is used to
avoid as long as possible a better performance by the child
because of imitation rather than spontaneous speech. If this
is not successful, the child will be asked to repeat the word
after the examiner who notes that the word is repeated, i.e.
the target consonant in the context of the word, which will
be taken into account during the analysis. In order to avoid
a slightly better performance of the target consonant,
phonetic prompting is not used under any circumstances.
The examiner repeats the target word after the child in
order to identify the target. A short pause between the
child’s production and adult’s is preferable for ease of
editing the speech samples at a later date. A syllable string
of /ma ma ma ma ma/ is repeated by the child, and finally
samples of connected speech are elicited using the Bus story
352 Cleft Palate–Craniofacial Journal, July 2009, Vol. 46 No. 4
TABLE 2 Cleft Palate Speech Characteristics
Active Characteristics
Anterior Oral Posterior Oral
Bilabial prepalatal
Labiodental palatal
Linguolabial velar
Interdental uvular
Postdental/alveolar anterior + velar or uvular (double articulation)
– language screen (Renfrew, 1997), as well as by
conversation.
Nasometry (Nasometer II 6400) is performed in the same
session. Standardized assessment protocols have been put
together for data collection. The nine high vowel words are
named in a string and repeated five times, each one saved
on the computer. The syllable string of /ma ma ma ma ma/
is also repeated five times and each time saved accordingly.
The repeated measures allow for calculation of error of
measurement. Finally, the language-specific sentences are
repeated and saved.
Background information regarding performed examina-
tions for velopharyngeal impairment (VPI), secondary
surgery, ear and hearing status, language screening, and
speech therapy are collected on separate proformae and
sent with copies of recordings and audiograms to the
project coordinator. The speech analysis will be performed
using a blinded procedure at a later date.
Analysis
The observer (SLP/T) carrying out the analysis should
speak the same language as the subject, i.e., observer and
the child’s language background will be the same. It is very
important that the observer does not analyze children from
his/her center to ensure independent and unbiased analysis.
Consonants
Narrow phonetic transcription of the target consonants
from video recordings will be performed according to the
conventions based on the IPA and the extended IPA for
disordered speech (ExtIPA) (International Phonetic Asso-
ciation, 1999). The child’s production of a target sound is
narrowly transcribed and compared to the target pronun-
ciation, i.e., the correct pronunciation given by an adult
speaker of the same language. Differences or ‘‘errors’’
between the two productions, termed cleft speech charac-
teristics (CSCs) are identified and categorized according to
a battery of these characteristics. This is repeated for all
target consonants, and the errors are recorded on an error
matrix type proforma. The battery of CSCs or ‘‘errors’’
used in the Scandcleft study is presented in Table 2 and is a
revised version of the battery suggested by Henningsson
and Hutters (1997) typical of cleft palate speech and similar
to GOS.SP.ASS. (Sell et al., 1999). The detailed analysis
Passive Characteristics
Posterior Nonoral Nasal Air Leakage Weakness
pharyngeal plosive nasal friction weak pressure consonants
pharyngeal fricative velopharyngeal friction nonaspiration
glottal plosive fistula friction
oral + glottal plosive (reinforcement)
nasal fricative
based on narrow phonetic transcription will always make it
possible to take a close look at the ‘‘raw data’’ and can also
be used to do a detailed phonetic presentation of data.
However, the reduction of data into CSCs is possible and
practical for communicating the results from a larger study
like the Scandcleft project. The frequency of CSCs for each
subject will be derived from a proforma showing transcrip-
tion of each of the target consonants produced by the child.
Nasal Resonance
Nasal resonance will be evaluated through perceptual
and instrumental analysis. Hypernasality is to be rated on
high vowels in single words, but the procedure is still to be
agreed. The first sample contains each one of nine high
vowels in nine single words, and the second connected
speech in retelling the Bus Story. Presence or absence of
hyponasal resonance on the /ma ma ma ma ma/ syllables
will be performed as well. Nasalance values are computed
repeatedly on the speech samples according to the
procedure described briefly above.
Reliability
The quality of the data will be measured by reliability,
e.g., by percent agreement. One hypothesis in the project
related to the reliability of transcription is that training will
increase the level of agreement. Training in phonetic
transcription of the target consonants produced by children
with cleft lip and palate will increase the agreement of what
the observer hears and how articulation can be described
and written down using the IPA and ExtIPA. Transcription
training on similar material to that collected for the
Scandcleft project has been undertaken by the large
Scandcleft Speech Group on three occasions between
2003 and 2007. Principles for training have been formulat-
ed. A transcription test was developed for the Scandcleft
Speech Group to be performed before each training session.
The results from these tests will be presented in a separate
article. Based on the experiences from the test and training,
rules for transcription in the project are being formulated
and include, e.g., ‘‘nasal emission diacritics are not possible
on voiced sounds; voiced sounds can phonetically not have
audible nasal airflow accompanying them,’’ ‘‘glottal sounds
cannot have nasal emission—accompanying signs of VPI.’’
These were used in the pilot assessment.
 Lohmander et al., SPEECH ASSESSMENT IN THE SANDCLEFT PROJECT 353
SCANDCLEFT PILOT ASSESSMENT
Pilot Project Material
Ten video recordings of children in each language in the
Scandcleft study were randomly selected by the central
administrator of the project (based in Manchester). The
recordings were sent to the Speech Group leader in
Gothenburg. One researcher (not involved in the pilot
assessment) prepared the video recordings for the study. Each
recording was digitized and loaded into the program Ulead
VideoStudio 9 (Corel Corporation, Ottawa, ON, Canada).
Each test word produced by the child and then repeated by the
examiner was saved as a separate video clip in mpeg2 format.
For English, Swedish, Danish, and Norwegian that meant 33
such files; for Finnish without initial /f/, 30 files were created.
The clips were numbered according to the numbers on the
word test proformae in the project. In addition, the children’s
retelling of the Bus Story and the syllable string /ma ma ma
ma ma/ were each saved as a video clip. All clips were stored in
document folders and saved on a DVD. On each DVD, 10
folders with 33/30 video clips in each language were saved.
Procedure
Five SLP/Ts who are involved in the Scandcleft project
and represent each of the five languages were invited to
participate as observers in the pilot study. The observers
were not from the same center as were the recordings,
except for the Finnish representative. Finland has only one
center, and it was therefore decided to select the recordings
of those children whom the invited observer had not been
involved with clinically, i.e., who were recorded before the
observer was employed.
On the first day of the pilot study, there was discussion
about the plan for the study. The linguistic consultant on the
project (B.H.) went through the test proforma and rules for
transcription based on the IPA and ExtIPA for disordered
speech (International Phonetic Association, 1999) and on
previous experiences from the transcription tests made in the
Scandcleft Speech Group in 2003 and 2005. All five observers
were familiar with the protocol for transcription and
assessment of cleft speech characteristics that have been
developed in the group since 1997. The second day was used
for assessment of the 10 children in each language. Each
observer blindly assessed speech of children in their own
language. A high-quality computer and headphones were
used for listening that took place in quiet rooms. A blinded
assessment scheme was used for the assessment performed
individually and is described in Analysis, below. The
observers were allowed to listen an unlimited number of
times to the material. After finishing one folder (for one
child) the observer took a short break, and in the middle of
the day, a lunch break. When all transcriptions were
completed, the observers judged the variables hypernasality,
overall velopharyngeal function, and hyponasality on the
same children starting from the beginning. A separate
proforma was designed for this part of the assessment. The
third day was used for completion and reassessment of all
material on two subjects for calculation of intraobserver
reliability. The last day was used for discussion about the
method for assessment as well as comments about the data
collection and documentation in the Scandcleft project.
Analysis
The following order was used for the blinded analysis in
the pilot project, further described below.
1. Single words
a. narrow phonetic transcription of the 33/30 target
consonants
b. rating of hypernasality
2. Connected speech (retelling the Bus Story)
a. rating of hypernasality
b. an overall judgment of perceived velopharyngeal
closure
3. Syllable string or single words
a. rating of hyponasality.
Consonants
The frequency of occurrence of CSCs for each subject
was obtained from the proformae with transcription of
target consonants. Each consonant was, if applicable,
translated by the observer into CSCs divided into the two
main categories—active and passive characteristics— by
marking a cross in the place of articulation and other
relevant features in the proforma. The ‘‘crosses’’ were
pooled into the following main categories: anterior oral,
posterior oral, posterior nonoral, nasal air leakage, and
weakness (Table 2). This means that all placement errors
that were bilabial, labiodental, linguolabial, interdental, or
postdental/alveolar were pooled into the anterior oral
category; errors that were prepalatal, palatal, velar, uvular,
or double velar/uvular were pooled into the posterior oral
category; and the errors in terms of glottal plosive,
pharyngeal fricative, pharyngeal plosive, the oral-glottal
plosive (reinforcement), and the nasal fricative were pooled
into the posterior nonoral category. This is the most deviant
category and is thought to be related to velopharyngeal
impairment. The presence of a CSC was based on the
number of consonants affected, i.e., three or more affected
5 present, and less than three 5 absent. This part of the
procedure is equivalent to the one used by John et al. (2006)
and was carried out by the first author.
Hypernasality
Hypernasality in each of nine high vowels in nine (high
vowel) single words was assessed along a five-point ordinal
354 Cleft Palate–Craniofacial Journal, July 2009, Vol. 46 No. 4
TABLE 3 Rating Scales for Hypernasality; I in High Vowels in
Single Words, and II in Connected Speech
Ratings Scale I: High Rating Scale II: Connected Speech
Score Vowels in Single Words (Retelling the Bus Story)
0 Absent absent
1 Slight/minimal slight/minimal
2 Mild mild – evident on high vowels
3 Moderate moderate – evident on high and low vowels
4 Severe severe – evident on vowels and voiced
consonants
scale. An additional rating scale according to the descrip-
tion by John et al. (2006) was used for assessment of
hypernasality in connected speech (Table 3).
Hyponasality
The presence or absence of hyponasality was judged on
the nasal consonant in the syllable string /ma ma ma ma
ma/ if available, otherwise the initial nasal consonants in
three single words were used.
Velopharyngeal Closure
In this pilot study of perceptual assessment, an overall
judgment of velopharyngeal closure was made from
connected speech using a three-point ordinal scale (Table 4)
to be used for validation of the velopharyngeal composite
score below. The Scandcleft project also includes instru-
mental analysis with nasometry, and visualization tech-
niques are used in any case of suspected VPI.
Velopharyngeal Composite Score – Summary (VPC-Sum)
A summary outcome measure was extrapolated from the
raw data (Table 5). Such a summary variable can
potentially be used as the main outcome variable consisting
of scores of posterior nonoral articulation, nasal air
leakage, weakness, and hypernasality—the primary indica-
tors on velopharyngeal impairment (Dotevall et al., 2002).
The substitution of posterior nonoral consonants for oral
consonants (compensatory articulation not requiring velo-
pharyngeal closure), nasal air leakage (audible nasal
emission and velopharyngeal friction sounds accompanying
consonants), weak pressure on pressure consonants, and
hypernasal resonance are the primary indicators for
insufficient velopharyngeal function (Trost, 1981). Hyper-
nasality was previously rated in connected speech along a
five-point ordinal scale, where 0 5 absent and 4 5 severe
hypernasality. Scale values 0 and 1 are considered to be
normal or a minor defect and gave a new score 0, while
scores 2, 3, and 4 are considered to be deviant giving a
new score 1. The number of target consonants with
symptoms (from transcriptions) of each of the variables
of nasal air leakage, weak pressure, and posterior nonoral
errors was calculated, and the sum given a new value. If
TABLE 4 Rating Scale for Perceived Velopharyngeal Closure
Score Velopharyngeal Closure Comment
0 Competent can include active nasal fricatives*
1 Marginally competent evidence of minor problems suggesting
borderline closure
2 Incompetent evidence of significant problems usually
requiring surgical management
* Particularly active nasal fricatives are often misinterpreted as symptoms of
velopharyngeal insufficiency (VPI) and are therefore mentioned.
three or more target consonants had symptoms of nasal air
leakage, the child was given a score of 1 on nasal air
leakage, etc. If fewer target consonants were affected, a
score 0 was given.
The VPC-sum was calculated from the previously
obtained score on each of the four variables. Zero (0)
means no or minor deficit and 1 a deficit. The VPC-sum
can therefore vary between 0 and 4. A VPC-sum of 0 to 1
means that none or only one of the included four variables
had a score of 1, and this is regarded as sufficient
velopharyngeal function. A VPC-sum of 2 means there is
a borderline deficit. A VPC-sum of 3 or 4 means that
velopharyngeal function is insufficient. The procedure is
equivalent to the one described in CAPS-A (John et al.,
2006), and a similar one was used in a recent study by
Lohmander et al. (2006).
RESULTS AND DISCUSSION
General
In the preassessment discussions, a consensus among the
observers was achieved on rules for transcription. Unfor-
tunately, some material was missing (occasionally naming
of the test words, more often retelling of the Bus Story, and
sometimes the /ma ma ma ma ma/ string) on the recordings,
and it was noted that the instructions for data collection
and documentation had not always been followed.
Each speech sample package for one child took 45 to
60 minutes to assess, including the assessment of resonance,
and most observers were able to do all of the listening tasks
within the stipulated amount of time. The observers
sometimes forgot to translate the transcriptions into CSCs
by making crosses in the relevant categories, which might
be explained both by the fact that the assessment was quite
tiring, and that they might have hurried in order to
complete the task.
Reliability and Feasibility
Reliability of the assessment was calculated as intraob-
server agreement, which was completed using data from the
phonetic transcription of the consonants, rating of hyper-
nasality on high vowels, rating of hyponasality on nasal
consonants, and overall rating of hypernasality and
velopharyngeal closure in connected speech.
 Lohmander et al., SPEECH ASSESSMENT IN THE SANDCLEFT PROJECT 355

TABLE 5 Velopharyngeal Composite Score Summary (VPC-Sum)

Composite Score Variables (From Pooled Data) Definitions Score 0 Score 1

Posterior nonoral articulation sum of posterior nonoral errors and active nasal fricatives 0–2 consonants affected 3 or more consonants affected
Nasal air leakage audible nasal friction or velopharyngeal friction sounds 0–2 consonants affected 3 or more consonants affected
accompanying consonants
Weakness reduced pressure on pressure consonants 0–2 consonants affected 3 or more consonants affected
Hypernasality rated on a five-point scale degree 0–1 degree 2–4

Transcription perceived weakness, but this realization will not be

Agreement of phonetic transcription is influenced by a
large number of factors. In this study, some of the most
obvious factors are the experience of the observer, degree of
narrowness, and occurrence of deviances in the speech
(Howard and Heselwood, 2002). Although all of the
observers were well experienced in analyzing cleft palate
speech, the mean point-by-point value for exact intraob-
server agreement (exactly the same symbol and diacritic)
varied quite a bit among the observers with a range from
27% to 88%. The transcriptions, therefore, were reviewed
regarding disagreements possibly due to factors other than
real disagreements, e.g., unfamiliarity with symbols. The
transcriptions were divided into those that could definitely
be explained by disagreements and those that possibly
could be explained by other factors, e.g., small allophonic
variations (Appendix 2). The process revealed intra-
agreement figures between 53% and 89% after changed
disagreements (Table 6).
Rules and definitions that have been formulated
throughout the years from experiences in the Scandcleft
project of transcription tests and training were found to be
valuable but still insufficient. Continuous experience from
testing and training is considered necessary to make the
procedure reliable. In this pilot material only a few
examples of CSCs were found, which could be due to lack
of narrowness in transcription. For example, if the obsevers
routinely do not notice nonaspiration as an error in
consonants, where the target consonant should be pro-
duced with aspiration, important differences in perceived
pressure that could be related to velopharyngeal function
might get lost. Aspiration is one important feature that
differs between the languages. The plosives /p t k/ are often
aspirated in English, Swedish, and Norwegian; although,
aspiration is not a distinctive feature. Nonaspiration in
these plosives may result in lack of distinctness or even
TABLE 6 Mean Intra-Agreement of Phonetic Transcriptions:
Agreements for Five Different Languages (for Details, See
Appendix 2)
interpreted as /b d g/. In Danish, aspiration is a distinctive
feature, and lack of aspiration on /p t k/ means that the
child’s production crosses a phonemic boarder, i.e., will be
identified as /b d g/. Finnish, on the other hand, has only /p
t k/, and their realization varies as to aspiration (and
voicing), but as opposed to the other languages, absence of
aspiration will never be interpreted as a symptom of VPI.
Thus, the narrowness of transcription is important and
might be decisive for the ability to identify differences in
speech related to surgical procedures. The detailed analysis
based on phonetic transcription, irrespective of how the
results will be presented, will always make it possible to
take a closer look at the ‘‘raw data.’’ However, collapsing
the data into CSCs is possible and practical for presenting
the results. The rows on the analysis proforma for marking
‘‘correct articulation’’ could give information on the
number of correct responses in addition to information
about errors.
Resonance
The intraobserver agreement on rating of hypernasality
on high vowels was found to be unacceptable with exact
agreement values ranging from 20% to 100% with a median
agreement of 50% (Table 7). This could be an effect of the
choice of using high vowels and a five-point scale in the
assessment. Hypernasality is easily perceived on high
vowels already with a small insufficiency in velopharyngeal
TABLE 7 Mean Values of Intra-Agreement of Hypernasality in
High Vowels in Single Words (Rating Scale I, Table 3), Connected
Speech (Rating Scale II, Table 3), and in Velopharyngeal Closure in
Connected Speech for Five Different Languages
Ordinal Scale Rating*
Hypernasality on Hypernasality Velopharyngeal
High Vowels in in Connected Closure in
Single Words Speech (Five-Point Connected Speech
(Five-Point Scale I) Scale II) (Three-Point Scale)
Language Median of Nine Mean Exact Exact
Number Values (Min-Max) Agreement Agreement

Language Exact (Unchanged) Agreements Including Changed 1 50% (20–80) 100% 100%
Number Agreements Disagreements 2 25% (20–30) 100% 100%
3 60% (30–60) 0%{ 100%
1 88% 89% 4 45% (30–60) 100% 100%
2 27% 53% 5 100% (100){ 100% 100%
3 73% 89%
4 86% 88% * For definitions of the rating scale, see Table 3.
5 68% 83% { Only one repeated rating with a difference of one scale value.
{ Only scale value ‘‘0’’ (normal) was used.
356 Cleft Palate–Craniofacial Journal, July 2009, Vol. 46 No. 4
closure. With a five-point scale this fact might lead to
inability or difficulties to perceive or differentiate degrees
other than presence or absence of hypernasal resonance,
and also to difficulties repeating the same assessment. One
alternative might be to rate hypernasality on the words with
high vowels in a string. The advantage of such a procedure
is that the result can be compared with a nasalance value
that is obtained on the same material. Further, a
dichotomized or a three-point rating scale might be
relevant. Before such a decision is made the reliability of
the procedure will be tested further on separate material.
The overall rating of hypernasality based on connected
speech was an easier task to perform, and the exact
agreement was 100% for four observers. One observer
reported only one repeated rating and that was within one
scale value. All five observers were 100% consistent in their
exact agreement on the rating of velopharyngeal closure
and on the presence or absence of hyponasality. As to
present/absent hyponasality, this resonance quality should
preferably be assessed in the nasal consonant occurring in
the syllable string since nasalance data are based on this
speech material. However, if this material was missing,
nasal consonants in single words were used for assessment
of hyponasality.
VPC-Sum and Velopharyngeal Closure
In order to investigate the validity of the results of the
VPC-sum calculation from single word data, the results were
compared with those of the traditional rating using the
kappa statistic and measuring the observed agreement.
Dotevall et al. (2002) also used a composite score to
determine the validity of perceptual ratings. In that study,
the composite score consisted of the sum of the perceptual
ratings of hypernasality, audible nasal escape, velopharyn-
geal friction sounds, and weak pressure consonants, and was
found to correlate very well with the perceptual rating of the
variable ‘‘velopharyngeal function.’’ In the present study, the
error variable ‘‘posterior nonoral articulation’’ was included
in the composite score as well. It could be argued that this
type of error in speakers with cleft palate is a consequence of
VPI but not necessarily an actual symptom (i.e., a persistent
error even though the VPI has been successfully treated) that
needs to be considered in the final procedure.
The observed agreement between the two methods for
measuring velopharyngeal closure was 78%. This figure was
calculated from data collected on 46 subjects (values were
missing for four children) from the 10 centers speaking five
languages in the Scandcleft project (Table 7). This shows
that the two methods for measuring velopharyngeal closure
agree on a high number of occasions. A kappa statistic and
95% confidence interval were also calculated using these
data. The corresponding figures were 0.58 (kappa) and 0.32
to 0.84 (confidence interval). The two-by-two table that was
used to calculate the kappa score contained one zero cell
count. That could lead to an underestimated effect, where
there is in fact good observed agreement, but the kappa is
low. Therefore, it is advised to pay more attention to the
actual observed agreement than to the kappa.
In the Scandcleft project, the results of velopharyngeal
function will be completed with results from investigations
with visualization techniques in children with suspected
VPI and any secondary surgeries performed.
Activity and Participation
No measures on the levels of activity and participation
are included in the assessment at age 5 years in the
Scandcleft project. The universal parameters by Henning-
sson et al. (2008) include two global parameters, ‘‘under-
standability’’ and ‘‘acceptability.’’ Both of these represent
the level of communicative activity and participation
according to the International Classification of Function-
ing, Disability, and Health (WHO, 2001). Other variables
in the various systems for reporting speech outcome are
based on analyses on the level of function. The need to
capture information on all levels was shown in a study on
palate re-repair, where patients who were perceptually
judged to have good speech, experienced restrictions in
communicative activity and participation (Sommerlad et
al., 2002). More recently, Havstam et al. (2008) found a
negative correlation between judgments of speech by
trained observers and the patients’ satisfaction with their
own speech. Communicative activity and participation (or
restrictions in) are typically assessed by analyzing speech
intelligibility and by questionnaires on the speakers’
satisfaction and experiences in different communicative
situations (Havstam et al., 2008).
FINAL COMMENTS AND CONCLUSIONS
The procedures for data sampling and recording seem
valid and reliable but have sometimes been hard to follow.
The methodology was developed during the project and
resources for equipment and personnel were not available
from the beginning. The result of this pilot project will be
used to remind the participants to continue to do their very
best to adhere closely to the protocol for data sampling and
recording, and for further development of method for
analysis. The challenges of pooling data of speakers of
different languages in the same trial and comparing speech
outcome across three trials are not as great as they might
sound given that the assessment of speech concerns
consonants and is confined to speech units that are
phonetically identical—or similar—across languages.
The results of the final assessments in the Scandcleft
project will be based on intralanguage and across-center
assessments. The complete material will consist of 150
children in each of the three trials. Since there are two
languages in each trial, over 75 analyses will be made at
each center. The pilot study has shown that it is possible to
achieve good consistency in terms of phonetic transcrip-
 Lohmander et al., SPEECH ASSESSMENT IN THE SANDCLEFT PROJECT 357
tion. However, interagreement between observers within
language needs to be tested. Furthermore, information and
continuous/ongoing training following agreed conventions
and rules are important. Training also needs to be ongoing
to ensure that any new members coming into the Scandcleft
Speech Group are trained, should they be involved in
carrying out the final analysis. This is particularly
important regarding the diacritics likely to be necessary
for distinguishing speech outcomes between trials involving
more languages.
A follow-up analysis of reliability of assessment of
hypernasality on high vowels and ongoing training will
hopefully finalize the method and prepare the observers for
the final assessment within the next couple of years. The
VPC-sum score based on the four main speech character-
istics of velopharyngeal impairment seems to have good
criterion validity shown by good agreement with overall
rating of velopharyngeal closure from connected speech.
The speech outcome of the Scandcleft project will be
based on the perceptual analyses of speech correlated with
the need for additional investigations of the velopharyngeal
function, secondary surgical interventions performed,
speech intervention, and status and intervention for ears
and hearing. A follow-up of speech at age 10 years has been
agreed upon in the Scandcleft project. We strongly
recommend adding measures of communicative activity
and participation, e.g., speech intelligibility, in order to
increase the general validity of the results.
Acknowledgments. We are indeed grateful to all speech and language
pathologists and therapists who have participated in the development of
the methodology and the data collection in the Scandcleft project
throughout the years.
REFERENCES
Brøndsted K, Grunwell P, Henningsson G, Jansonius K, Karling J, Meijer
M, Ording U, Sell D, Vermeij-Zieverink E, Wyatt R. A phonetic
framework for the cross-linguistic analysis of cleft palate speech. Clin
Linguist Phon. 1994;8:109–125.
Coussé E, Gillis S, Kloots H, Swerts M. The influence of the labeller’s
regional background on phonetic transcriptions: implications for the
evaluation of spoken language resources. In: Proceedings of the
International Conference on Language Resources and Evaluation.
Lisbon, Portugal, 2004:1447–1450.
Dalston RM, Marsh JL, Vig KW, Witzel MA, Bumsted RM. Minimal
standards for reporting the results of surgery on patients with cleft lip,
cleft palate, or both: a proposal. Cleft Palate J. 1988;25:3–7.
Dotevall H, Lohmander A, Ejnell H, Bake B. Perceptual evaluation of
speech and velopharyngeal function in children with and without cleft
palate and the relationship to nasal airflow patterns. Cleft Palate
Craniofac J. 2002;39:409–424.
EUROCRAN Speech Project. Available at: http://www.eurocran.org.
Accessed February 11, 2008.
Fletcher SG, Bishop ME. Measurement of nasality with TONAR. Cleft
Palate J. 1970;7:610–621.
Fujiwara Y, Henningsson G, Ainoda N. A review of Japanese articles on
perceptual assessment of speech sounds in individuals with cleft palate.
Jpn J Logoped Phoniatr. 2006;47:252–257.
Gooch JL, Hardin-Jones M, Chapman KL, Trost-Cardamone JE,
Sussman J. Reliability of listener transcriptions of compensatory
articulations. Cleft Palate Craniofac J. 2001;38:59–67.
Grunwell P, Sell D, Harding A. Describing cleft palate speech. In:
Grunwell P, ed. Analysing Cleft Palate Speech. London: Whurr; 1993.
Harding A, Grunwell P. Active versus passive cleft-type characteristics:
implications for surgery and therapy. Int J Lang Commun Disord.
1998;33:329–352.
Harding A, Harland K, Razzell R. Cleft Audit Protocol for Speech
(CAPS). Broomfield, Chelmsford, Essex: St. Andrew’s Plastic Surgery
Centre; 1997.
Havstam C, Lohmander A, Dahlgren Sandberg A, Elander A. Speech and
satisfaction with treatment outcome in young adults with unilateral or
bilateral complete clefts: answers to a patient questionnaire compared
to speech assessments. Scand J Plast Reconstr Surg Hand Surg.
2008;42:182–189.
Henningsson G, Hutters B. Perceptual speech assessment of cleft palate
speech—with special reference to minimum standards for inter-centre
comparisons of speech outcome. In: Transactions of the 8th Interna-
tional Congress on Cleft Palate and Related Craniofacial Anomalies.
Singapore; 1997:xxxiii–xxxvii.
Henningsson G, Isberg A. Oronasal fistulas and speech production. In:
Bardach J, Morris H, eds. Multidisciplinary Management of Cleft Lip
and Palate. Philadelphia: WB Saunders; 1990:787–792.
Henningsson G, Kuehn DP, Sell D, Sweeney T, Trost-Cardamone J,
Whitehill T. (Speech Parameters Group). Universal parameters for
reporting speech outcomes in individuals with cleft palate. Cleft Palate
Craniofac J. 2008;45:1–17.
Howard SJ, Heselwood BC. Learning and teaching phonetic transcription
for clinical purposes. Clin Linguist Phon. 2002;16:371–401.
Hutters B, Brøndsted K. Strategies in cleft palate speech—with special
reference to Danish. Cleft Palate J. 1987;24:126–136.
Hutters B, Henningsson G. Speech outcome following treatment in cross-
linguistic cleft palate studies: methodological implications. Cleft Palate
Craniofac J. 2004;41:544–549.
International Phonetic Association. Handbook of IPA. Cambridge, U.K.:
Cambridge University Press; 1999.
John A, Sell D, Sweeney T, Haring-Bell A, Williams A. The cleft audit
protocol for speech – augmented: a validated and reliable measure for
auditing cleft speech. Cleft Palate Craniofac J. 2006;43:272–288.
Kent RD. Hearing and believing: some limits to the auditory-perceptual
assessment of speech and voice disorders. Am J Speech Lang Pathol.
1996;5:7–23.
Keuning KHD, Wieneke GH, Dejonckere PH. The intrajudge reliability
of the perceptual rating of cleft palate speech before and after
pharyngeal flap surgery: the effect of judges and speech samples. Cleft
Palate Craniofac J. 1999;36:328–333.
Kreiman J, Gerratt BR, Kempster GB, Erman A, Berke GS. Perceptual
evaluation of voice quality: review, tutorial, and a framework for
future research. J Speech Hear Res. 1993;36:21–40.
Kuehn D, Moller K. Speech and language issues in the cleft palate
population: the state of the art. Cleft Palate Craniofac J. 2000;37:1–35.
Kuehn DP, Moon JB. Velopharyngeal closure force and levator veli
activation levels in varying phonetic contexts. J Speech Lang Hear Res.
1998;41:51–62.
Lewis KE, Watterson TL, Houghton SM. The influence of observer
experience and academic training on ratings of nasality. J Commun
Disord. 2003;36:49–58.
Lintz LB, Sherman D. Phonetic elements and perception of nasality.
J Speech Hear Res. 1961;4:381–396.
Lohmander A, Borell E, Henningsson G, Havstam C, Lundeborg I,
Persson C. SVANTE - Svenskt Artikulations- och Nasalitets Test
(Swedish Articulation and Nasality Test) [in Swedish]. Skivarp,
Sweden: Pedagogisk Design; 2005.
Lohmander A, Friede H, Elander A, Persson C, Lilja J. Speech
development in patients with unilateral cleft lip and palate treated
with different delays of hard palate closure after early velar repair: a
358 Cleft Palate–Craniofacial Journal, July 2009, Vol. 46 No. 4
longitudinal perspective. Scand J Plast Reconstr Surg Hand Surg.
2006;46:267–274.
Lohmander A, Olsson M. Perceptual assessment of speech in patients
with cleft palate: a critical review. Cleft Palate Craniofac J. 2004;41:
64–70.
McWilliams BJ, Morris LM, Shelton RL. Cleft Palate Speech. Philadel-
phia: BC Decker; 1990.
Renfrew C. The Renfrew Language Scales: Bus Story Test—A Test of
Narrative Speech. Bicester, Oxon: Winslow Press Limited; 1997.
Scandcleft Trials Operational Manual. University of Manchester; 1997.
Sell D. Issues in perceptual speech analysis in cleft palate and related
disorders: a review. Int J Lang Commun Disord. 2005;40:103–121.
Sell D, Harding A, Grunwell PA. GOS.SP.ASS’98: an assessment for
speech disorders associated with cleft palate and/or velopharyngeal
dysfunction (revised). Int J Lang Commun Disord. 1999;34:17–33.
Shaw WC, Semb G, Nelson P, Brattström V, Mølsted K, Prahl-Andersen
B. The Eurocleft Project 1996–2000. Standards of Care for Cleft Lip and
Palate in Europe. European Commission Biochemical and Health
Research. Amsterdam: IOS Press; 2000.
Shriberg LD, Lof GL. Reliability studies in broad and narrow phonetic
transcription. Clin Linguist Phon. 1991;1:171–189.
Sommerlad BC, Mehendale FV, Birch MJ, Sell DA, Hattee C, Harland K.
Palate repair revisited. Cleft Palate Craniofac J. 2002;39:295–307.
Spriestersbach DC, Powers GR. Nasality in isolated vowels and connected
speech of children with cleft palate. J Speech Hear Res. 1959;2:40–45.
Trost JE. Articulatory additions to the classical description of the speech
of persons with cleft palate. Cleft Palate J. 1981;18:193–203.
Williams WN, Seagle MB, Nackashi AJ, Marks R, Boggs SR, Kemker J,
Wharton W, Bzoch KR, Dixon-Wood V, Pegoraro-Krook MI, et al. A
methodology report of a randomized prospective clinical trial to assess
velopharyngeal function for speech following palatal surgery. Control
Clin Trials. 1998;19:297–312.
World Health Organization. International Classification of Function,
Disability and Health. Geneva: WHO; 2001.
 Lohmander et al., SPEECH ASSESSMENT IN THE SANDCLEFT PROJECT 359

APPENDIX 1
The test words in each language in the Scandcleft project. In the appendix, the words are ordered according to place of articulation.
In elicitation, the words are presented in random order.
360 Cleft Palate–Craniofacial Journal, July 2009, Vol. 46 No. 4

APPENDIX 1
Continued
 Lohmander et al., SPEECH ASSESSMENT IN THE SANDCLEFT PROJECT 361

APPENDIX 2
Overview of the disagreements from Scandcleft pilot transcription assessment.
362 Cleft Palate–Craniofacial Journal, July 2009, Vol. 46 No. 4

APPENDIX 2
Conitnued