ARTICLE IN PRESS

Validation of Acoustic Voice Quality Index Version 3.01 and

Acoustic Breathiness Index in Korean Population
*Geun-Hyo Kim, †Ben Barsties von Latoszek, and *Yeon-Woo Lee, *Busan, South Korea, and yAntwerp, Belgium

Summary: Objectives. This study aimed to verify the Acoustic Voice Quality Index (AVQI) version 3.01 and
the Acoustic Breathiness Index (ABI) as tools for acoustic analyses in the Korean language.
Methods. Concatenated voice samples of sustained vowels (SV) and continuous speech (CS) were collected
from 151 subjects with dysphonia and 71 vocally healthy subjects. The overall voice disorder severity (grade [G]
and overall severity [OS]) and breathiness severity (B) were subjected to an auditory-perceptual rating by three
raters. First, we equalized the proportions of SV and CS with respect to the time lengths of the voice samples to
improve the ecological validity. We then validated the AVQI and ABI in the Korean language, using our most
recent dataset of 1,667 voice samples. Second, we compared the results of the acoustic analyses between the
vocally healthy controls and the dysphonia groups. Third, we confirmed the concurrent validity and diagnostic
accuracy using the Spearman rank-order correlation coefficient (rs) and various statistical methods (receiver oper-
ating characteristic curve, pairwise comparison, and likelihood ratio [LR] analyses).
Results. We observed strong inter-rater reliability for G, B, and OS. Moreover, we identified 26 standardized
syllables in the CS samples (3 second voiced segments), which allowed the equalization of both voice tasks.
A comparison of the two voice groups revealed statistically significant differences in the AVQI, ABI, G, B, and
OS (all P < 0.01). Moreover, we identified strong correlations of the AVQI with G (rs > 0.88, P < 0.01) and OS
(rs > 0.84, P < 0.01) and of ABI with B (rs > 0.87, P < 0.01). Finally, we confirmed cutoffs of 3.154 (sensitivity:
90%, specificity: 89%, LR+: 8.45, and LR-: 0.12) and 3.685 (sensitivity: 88%, specificity: 86%, LR+: 6.47, and
LR-: 0.14) as optimal predictive powers for AVQI and ABI, respectively.
Conclusion. As per our results, in a sample of Korean speakers, the AVQI and ABI exhibited strong concurrent
validity for the quantification of dysphonia severity with respect to OS and B. We consider that analyses based on
the AVQI and ABI will enable the discrimination and assessment of dysphonia in clinical practice.
Key Words: Acoustic voice quality index−Acoustic breathiness index−Dysphonia−Auditory-perceptual rat-
ings.

INTRODUCTION Acoustic methods, such as two recently introduced acous-

Both clinical field and voice studies perceive voice as a mul-
tidimensional entity. Vocal quality is considered to be an
important factor in the evaluation of voice anomalies in
patients with dysphonia, such as roughness and hoarseness.1
Various types of voice evaluations have been used to under-
stand and confirm this multidimensional nature. Generally,
voice quality is assessed using acoustic, aerodynamic, and
video-endoscopic (visual) evaluations; auditory-perceptual
(A-P) ratings; and self-rated questionnaires.2 Standardized
A-P measures, such as the Grade, Roughness, Breathiness,
Asthenia, Strain (GRBAS) scale, and Consensus Auditory-
Perceptual Evaluation of Voice (CAPE-V), are considered
the reference standards for rating the severity of voice disor-
der.3,4 However, these measures are subjective, with limited
validity and intra- and inter-rater reliability.
Accepted for publication October 10, 2019.
Declarations of interest: none.
Financial disclosure: none.
From the *Department of Otorhinolaryngology-Head and Neck Surgery and Bio-
medical Research Institute, Pusan National University Hospital, Busan, South Korea;
and the yFaculty of Medicine and Health Sciences, University of Antwerp, Antwerp,
Belgium.
Address correspondence and reprint requests to Yeon-Woo Lee, Department of
Otorhinolaryngology-Head and Neck Surgery and Biomedical Research Institute,
Pusan National University Hospital, 179 Gudeok-ro, Seo-gu, Busan, South Korea.
E-mail: ahaha1216@gmail.com
Journal of Voice, Vol. &&, No. &&, pp. &&−&&
0892-1997
© 2019 The Voice Foundation. Published by Elsevier Inc. All rights reserved.
https://doi.org/10.1016/j.jvoice.2019.10.005
tic indexes with sufficient validity and reliability in voice
evaluations, could potentially overcome the limitations of
A-P ratings. First, the Acoustic Voice Quality Index
(AVQI) is a multiparametric index that measures the overall
voice quality using six acoustic parameters.5 The AVQI is
used to calculate data for a sustained vowel (SV; e.g., [a:])
and continuous speech (CS), which are combined for the
index measurements. The inclusion of both the SV and CS
increases the ecological validity of the AVQI by enabling a
better representation of the quality of voice used in a general
communication situation. An AVQI measurement may con-
nect the SV and CS and thus, include both types of speaking
samples, and a single score is calculated by measuring
concatenated speech samples using the PRAAT script.6
Several studies have reported a relatively strong relation-
ship between the AVQI and A-P ratings of overall voice
quality in various spoken languages. The regression equa-
tions used to calculate the AVQI were developed up to the
first, second, and third version. The first version measured
variables using two programs, such as the PRAAT and
SpeechTool programs,5 whereas only PRAAT was used to
measure all variables in the second version.7 The third ver-
sion of the AVQI (AVQIv3) was designed such that the
voice quality was analyzed after ensuring equal durations of
the SV and CS.8 The use of PRAAT to customize the AVQI
has increased the validity of the latter. Moreover, unlike the
 ARTICLE IN PRESS
2 Journal of Voice, Vol. &&, No. &&, 2019
Dysphonia Severity Index (DSI), which comprises the
parameters of Fo-high, intensity-low, jitter, and maximum
phonation time (MPT),9 the AVQI is not affected by age or
gender and is sensitive to changes in vocal recovery and
damage. The DSI includes the F0-high and MPT, affected
by gender and age. As the values of these variables increase,
the DSI value also increases. There is a significant difference
in the F0-high between males and females. In addition, rela-
tively low DSI values can be measured, especially in chil-
dren and the elderly, due to lack of breathing support (short
MPT). These attributes affect DSI resulting in low DSI val-
ues, even though the voice quality is not bad. These varia-
bles are not considered in the regression formula for DSI;
the DSI value may vary depending on gender and age,
regardless of the voice quality.
The second acoustic method, the Acoustic Breathiness
Index (ABI), is also used to analyze concatenated voice
samples of SV and CS and thus evaluate the severity of
breathiness,10 a parameter characterized by turbulent noise.
Breathiness is caused by the incomplete closure of the vocal
folds during phonation and is a prominent sign of laryngeal
diseases associated with glottal gaps, such as vocal fold nod-
ules, polyps, and vocal cord paralysis caused by recurrent
laryngeal nerve injury. This quality, defined elsewhere as
“audible unintentional air leakage in the voice”4, does not
occur independently of other characteristics such as asthenia
or roughness1 and is often attributed to increased airflow.11
Like the AVQI, the ABI is a multiparameter model and is
composed of nine acoustic variables. The ABI and AVQIv3
are similar with respect to the length of the voice samples
and rating systems used during analyses.12-18 According to
initial studies, the ABI yielded acceptable concurrent valid-
ity and predictive power in the Dutch,5 Spanish,13 Ger-
man,16 and Japanese languages.18 Although verified in
other languages, it is necessary to validate it in Korean lan-
guage. For example, between Korean and English, the for-
mer is a syllable-timed language with insignificant stress on
individual words, whereas the latter is a stress-timed lan-
guage.19 In other words, the rhythm and intonation of
Korean language are based on each syllable, whereas
English derives these from the distribution of stressed and
unstressed syllables. Moreover, the two languages differ in
the number of consonants and vowels. Korean language has
24 letters (14 consonants and 10 vowels), whereas English
language has 26 letters (21 consonants and 5 vowels).20 In
addition, English sounds, such as /f/, /v/, /th/, and /z/, are
not present in the Korean inventory, which has led to the
substitution of these sounds with the most similar ones.21
Sound composition in different languages can have different
effects on acoustic analysis. Therefore, it is necessary to vali-
date the standard tests in each language.
In this study, we aimed to verify both the AVQI and ABI
in a sample of Korean speakers and to determine the ability
of both indexes to discriminate vocal pathology. We have
verified diverse multivariate models that can quantify dys-
phonia and help us discriminate voice disorders. This study
included the equalization of SV and CS lengths to improve
the ecological validity, automatic analysis of concatenated
voice samples using the PRAAT script, large data sets, a rig-
orous selection process, analysis of intra- and inter-rater
reliability, and various validation studies.
The specific questions addressed in this study were as
follows:
1) What is the number of standardized syllables that cor-
respond to 3-second of vowel /a/ in Korean?
2) Are there significant differences between vocally
healthy and pathological voice groups in acoustic vari-
ables (AVQI and ABI) and A-P ratings (G, B, and
OS)?
3) What are the correlations between acoustic variables
and A-P ratings?
4) What are the optimal cutoff values and diagnostic pre-
dictive powers that can discriminate normal and patho-
logical voices?
MATERIALS AND METHODS
Subjects
This retrospective study included all voice samples obtained
from March 1, 2015 to February 28, 2019. The voice sam-
ples were recorded during routine voice assessments in a
voice clinic prior to medical intervention or behavioral ther-
apy. Subjects with voice disorders were recruited to partici-
pate in the present study at Pusan National University
Hospital in Korea. The Institutional Review Board of
Pusan National University Hospital approved this study
[H1904-011-078]. The initial study population comprised
voice samples from 253 subjects, of which 31 were excluded
following a signal to noise ratio (SNR) analysis. The 222
subjects after this exclusion were randomly selected, and the
severity of the voice was evaluated by three speech-language
pathologists (SLPs). We included subjects with organic,
functional, and neurological voice disorders who exhibited
dysphonia at varying levels of severity. The subjects
included 112 men and 110 women with a mean age
(§standard deviation [SD]) of 54.1 § 14.6 years (range:
16−87 years). Medical diagnoses were determined based on
chart reviews, patient interviews, laryngeal videoendoscopy,
and stroboscopy examinations.
The core aim of the study was to validate the ability of the
AVQI and the ABI in the Korean language to discriminate
between vocally healthy subjects and patients with dyspho-
nia. The vocally healthy group comprised subjects with no
pathological laryngeal lesions or voice disorders and a grade
of 0 on the GRBAS scale. In addition, there were no hearing
loss, communication problems, or neurological problems
(brain infarction, cerebral hemorrhage, Parkinson's disease,
etc.). The vocally healthy group was selected according to
the above criteria from those who visited the ENT clinic for
a medical check-up. No laryngeal disease was confirmed
through several evaluations such as laryngoscopy, patient
reports, and chart review.
 ARTICLE IN PRESS
Geun-Hyo Kim, et al Validation of Acoustic Voice Quality Index Version 3.01 and Acoustic Breathiness
Study design
Our research design has modified the method of research
presented in the previous study.12 Through the initial 222
samples (phase 1), we identified the corresponding syllable
in response to vowel /a/ of 3-second duration, and based on
this result, intra-/inter-rater reliabilities were confirmed for
A-P ratings (phase 2a). Subsequently, a large number of
voice samples (1,445 samples) were added for acoustic anal-
ysis (AVQI and ABI). For the correlations between acoustic
measurements and A-P ratings, A-P ratings were performed
on the entire voice samples (1,667 samples), and A-P rating
results of three people were presented as averaged scores. A
total of 1,667 voice samples were automatically analyzed
for AVQI and ABI through the PRAAT script, and the
results were saved as text files (phase 2b).
Voice samples
We assessed the voice samples according to the instructions
of the European Laryngological Society.22 We recorded all
voice samples using a cardioid AKG Perception 220 micro-
phone (AKG Acoustics, Vienna, Austria) situated at a fixed
interval of 10 cm from the lips and an angle of 45° from the
front of the mouth. The recordings were saved as WAV for-
mat files with a 44.1-kHz sampling rate and 16-bit quantiza-
tion using the Computerized Speech Lab (model 4500,
KayPENTAX, Lincoln Park, NJ). The subjects were
instructed to sustain vowel /a/ for more than 3 second and
read a phonetically balanced Korean text “Walk” with 70
syllables (/nopʰɯn sʰane ollaga malgɯn goEgirɯl maɕimyʌ
sʰorirɯl tsirɯmyʌn kasʰɯmi hwalts̕ aE ɲyʌʎʎinɯn tɯtʰada/
and /nʌlk̕ e pʰyʌltsʰʌinnɯn padarɯl parabomyʌn nɛ maɯm
yʌk̚ ɕ̕ i nʌlbʌdzinɯn gʌt̚ kat̚t̕ a/) in comfortable pitch and
loudness.
Signal to noise ratio (SNR)
We measured the SNR to ensure the reliability of the sound
recording. Previous studies have shown that stability of the
recording is guaranteed at a minimum SNR of 30 dB,
although a value >42 dB is preferred.23,24 The SNRs of the
recordings ranged from 19.8 to 57.2 dB, with a mean value
of 39.5 dB. The records of 31 subjects (from the pathological
voice group) with SNRs <30 dB were excluded from this
study.
Phase 1: Standardized syllable number in the CS
segments
To improve the ecological validity, the ratio of the two voice
processing segments (3-second SV and CS) must be bal-
anced. The numbers of standardized syllables (SSN) in the
CS segments of the Korean language samples were mea-
sured as proposed by Barsties and Maryn,25 who recom-
mended the following protocol for standardizing the CS
duration. First, we extracted all voiceless components of the
whole Korean sentence using the PRAAT script.6 Second,
we identified a hand-marked (HM) custom cutoff point in
3
the original text without removing the unvoiced interval
corresponding to the first 3-second of the 222 voice samples
containing only voiced segments. An acceptable value for
the SSN requires statistically no significant differences
between (1) the HM syllables and SSN and (2) the AVQI
and ABI values.
Phase 2a: Auditory-perceptual assessments
During the A-P assessments, the voice quality was rated by
a panel of three SLPs with 7−25 years of specialized profes-
sional experience in voice evaluation, therapy, and speech
science. The A-P assessments were rated by concatenating
the mid-portion of the SV (3-second) and the CS (i.e., the
final syllable number determined in Phase 1). The GRBAS
scale and CAPE-V were used for the A-P ratings. Each
rater scored the overall voice quality, which is represented
by the grade (G)-variable of GRBAS, the overall severity
(OS)-variable of CAPE-V, and the breathiness severity (B-
variable). The B-variable corresponds to the degree of audi-
ble or excessive emission of breath. The G and B variables
were scored using an ordinal four-point equalized interval
scale, whereas the OS variable was scored using a 100-mm
visual analog scale with anchoring points. In many previ-
ous studies, A-P ratings were conducted mainly using G
and OS scales. In this study, we used the B of GRBAS scale
to validate the ABI in Korean speakers according to the
method of research suggested in preceding studies.10,18
We did not evaluate the subitems of CAPE-V: breathiness
(B) because we tried to validate ABI in more representative
OS variables first, and then decided whether B would be
appropriate.
The A-P ratings were conducted in a quiet, noise-con-
trolled room, and all judgments were completed in a single
session. All raters were allowed to rate each voice sample as
many times as necessary before completing the final rating.
Completion times of 222 sample ratings were not measured.
To minimize factors such as fatigue and reduced concentra-
tion due to repeated A-P ratings, raters took short breaks
for every 50 voice samples. The recordings of 110 randomly
selected voice samples (»50% of the initial total samples)
were evaluated a second time after a 2-week interval to
assess the intra-rater reliability. Moreover, the A-P ratings
produced by the three raters were compared to measure
inter-rater reliability. A Google survey for the GRBAS vari-
ables G and B and a customized computer program for the
OS variable of the digital CAPE-V were used to facilitate
the saving of data files and analyses and enabled the auto-
matic saving of the A-P scores of the voice samples. The OS
rating program was designed for our research purposes and
is not a commercialized program. We already introduced it
in our previous study.26 This software will be complemented
and shared.
Phase 2b: Acoustic measures
Both the AVQI and ABI were applied to the concatenated
files comprising the 3-second SV and extracted CS voiced
 ARTICLE IN PRESS
4 Journal of Voice, Vol. &&, No. &&, 2019

TABLE 1. We used the next equation, Eq. (2), to calculate the

The Primary Laryngological Diagnoses Included in the ABI10:
Present Study
ABI ¼ ð5:0447730915 ½0:172  CPPS ½0:193  Jit
Age Absolute % of
Laryngeal status (Mean § SD) number total  ½1:283  GNEmax4; 500Hz
Vocally healthy 51.8 § 13.0 197 11.8%
Nodules 55.9 § 15.7 250 15.0%  ½0:396  Hfno6; 000Hz þ ½0:01  HNRD
Sulcus 190 11.4%
Cancer 186 11.2% þ ½0:017  H1H2 þ ½1:473  SLdB
Presby 177 10.6%
SD 117 7.0%  ½0:088  SL ½68:295  PSDÞ
Palsy 112 6.7%
Polyp 79 4.7%  2:9257400394:
PTC 75 4.5%
LRPD 66 4.0%
Papilloma 59 3.5%
Leuko 49 2.9% Statistics
Edema 47 2.8% The R platform, version 3.5.2 (The R Foundation for Statis-
Cyst 46 2.8% tical Computing, Vienna, Austria) and R Studio 1.1.463 (R
FD 17 1.0% Studio Inc., Boston, MA) were used to calculate all statisti-
Total 1,667 100.0%
cal variables except the inter- and intra-rater reliabilities.
Abbreviations: Cancer, glottic cancer; Cyst, vocal cyst; FD, functional dys- The latter variables were calculated using SPSS for Win-
phonia; Leuko, leukoplakia; Edema, laryngeal edema; LPRD, laryngophar-
yngeal reflux disease; Nodules, vocal nodules; Palsy, vocal cord palsy; dows, version 20.0 (IBM Corp, Armonk, NY). A P value of
Papilloma, laryngeal papillomatosis; Polyp, vocal polyp; Presby, presby- <0.05 was considered statistically significant.
laryngis; PTC, papillary thyroid cancer; SD, spasmodic dysphonia; Sul- First, we identified and set the SSN of the CS segment
cus, sulcus vocalis.
using rounded integers corresponding to the 95% confidence
interval lower bound of the HM syllable number, as
described by Barsties and Maryn.25 The Wilcoxon signed-
rank test was used to compare the AVQI and ABI values
between the cutoff point of the CS portion of the SSN and
segment. An additional 1,445 subjects (758 male and 687 the number of HM syllables.
female) were included to enhance the validity of the acoustic Second, to verify Phase 2a, we used the intraclass correla-
analyses (AVQI and ABI), and all corresponding samples tion coefficient (ICC) to calculate the inter-rater reliability
were assessed using the PRAAT script and the results were in G, B, and OS. Previous studies have reported that the
saved as text files.6,10 The subjects' data are summarized in GRBAS scale is an ordinal scale and ICCs are suitable for
Table 1. The following acoustic variables were measured. measuring the intra-/inter-rater reliability of ordinal scale
First, the six parameters of the AVQI were determined: (1) with two or more raters.27-32 The intra-rater reliability of
smoothed cepstral peak prominence (CPPS), (2) harmonics the three raters for the 110 voice samples was measured.
to noise ratio (HNR), (3) shimmer local (SL), (4) shimmer The scores of the three SLPs were averaged, and the results
local dB (SL-dB), (5) general slope of the spectrum (Slope), were expressed as mean G, B, and OS values for the 222
and (6) trend line of slope (Tilt). Second, the nine parame- subjects. ICC was calculated using a two-way mixed, consis-
ters of the ABI were determined: (1) CPPS, (2) jitter local tency, average-measures (P < 0.05).33
(Jit), (3) glottal to noise excitation ratio at 4,500 Hz (GNE- Third, we used an independent t test to evaluate the dif-
max-4,500Hz), (4) parameters of relative level of high-fre- ferences in acoustic analyses (AVQI and ABI) and A-P rat-
quency noise between energy (from 0 to 6,000 Hz and ings (G, B, and OS) between the vocally healthy controls
energy from 6,000 to 10,000 Hz) [Hfno-6,000Hz], (5) HNR and dysphonia groups (P < 0.05).
of Dejonckere (HNR-D), 6) differences between the ampli- Fourth, the concurrent validity of the AVQI and ABI was
tudes of H1 and H2 on the spectrum (H1-H2), (7) SL, (8) determined using the correlation coefficient (r) and coeffi-
SL dB, and (9) standard deviation of the period (PSD). cient of determination (r2). The Spearman rank-order corre-
In this study, we used the following equation, Eq. (1), to lation coefficients (rs) were measured between G, OS, and
calculate the AVQIv38: AVQI values and between B and ABI values (P<0.01).
AVQI ¼ ð4:152 ½0:177  CPPS ½0:006  HNR Correlation coefficient [r] is classified as follows: r ≥ 0.9,
very high; 0.7 ≤ r < 0.9, high; 0.5 ≤ r < 0.7, moderate;
 ½0:037  SL þ ½0:941  SLdB 0.3 ≤ r < 0.5, low; and r < 0.3, negligible.34
Fifth, we performed a receiver operating characteristic
þ ½0:01  Slope þ ½0:093  TiltÞ  2:8902 (ROC) curve analysis to calculate the cutoff values and diag-
nostic predictive accuracy (i.e., area under the curve [AUC])
 ARTICLE IN PRESS
Geun-Hyo Kim, et al Validation of Acoustic Voice Quality Index Version 3.01 and Acoustic Breathiness 5

of the AVQI and ABI for discriminating between the vocally TABLE 3.
healthy controls and pathological voice group. The ROC Intra-Rater Reliability of the Three SLPs Who Evaluated
curve analysis was also used to calculate the sensitivity G, B, and OS in A-P Ratings
(Sens.), specificity (Spec.), AUC, and likelihood ratio (posi-
ICC Rater 1 Rater 2 Rater 3
tive/negative result; LR+/LR-) of each acoustic measure
(AVQI or ABI) for discriminating pathological voice. G 0.949** 0.909** 0.910**
Finally, we applied a pairwise comparison to the ROC curves B 0.915** 0.892** 0.881**
and thus compared the AUCs of variables contributing to OS 0.873** 0.904** 0.901**
variance in the index parameters.35 Discriminative ability is ** P < 0.01.
Abbreviations: B, breathiness; G, gradel; ICC, intraclass correlation coeffi-
classified as follows: AUC ≥ 0.9, excellent; 0.8 ≤ AUC < 0.9, cient; OS, overall severity.
good; 0.7 ≤ AUC < 0.8, fair; and AUC < 0.7, poor.36

analyses and the A-P ratings according to groups. The

RESULTS
Standardized syllable number for the CS segment
In the CS segment, 10−68 syllables were required to achieve
a 3-second duration. We confirmed a SSN of 26 syllables as
the lower 95% confidence boundary. We did not observe
any significant difference in the duration of the HM syllable
selection and the SSN selection of 26 syllables (z = 1.060,
P = 0.289) (see Table 2). Furthermore, comparisons of the
AVQI and ABI values calculated using the HM syllable and
SSN with 26 syllables did not yield statistically significant
differences (AVQI: z = 1.372, P = 0.170; ABI: z = 1.043,
P = 0.297) (see Table 2). For further analyses, the CS seg-
ment was edited based on the SSN of 26.
Reliability of auditory-perceptual judgments
Tables 3 and 4 summarize the intra- and inter-rater reliabil-
ities of the A-P ratings produced by the three raters. The
intra-rater reliabilities for the G, B, and OS variables ranged
from 0.909 to 0.949, from 0.881 to 0.915, and from 0.873 to
0.904, respectively. The inter-rater reliabilities ranged from
moderate (ICC = 0.777 between raters 1 and 3 for G) to high
(ICC = 0.879 between raters 1 and 2 for G), with a mean ICC
of 0.833. The multiple correlation coefficients indicated strong
correlations among the three A-P parameters (G: 0.936, B:
0.930, and OS: 0.933). Taken together, these statistics indicate
statistically significant agreement in the A-P ratings among
the three SLPs. The general variability among the three raters
was considered acceptable for the purpose of this study.
Comparison of ABI and AVQI between two groups
Figure 1 presents the mean values measured for the acoustic
variables, while Table 5 lists the average values of acoustic
results demonstrate the statistically significant differences
confirmed for all variables (P < 0.01). Here, the vocally
healthy group exhibited significantly low values of AVQI,
ABI, G, B, and OS, whereas the dysphonia group exhibited
relatively high values.
Correlation between acoustic analyses (ABI and
AVQI) and A-P ratings (G, B, and OS)
First, we observed strong correlations of the AVQI with G
(rs > 0.88, P < 0.01) and OS (rs > 0.84, P < 0.01). Second,
we observed a strong correlation of the ABI with B (rs >
0.87, P < 0.01), indicating a high level of concurrent valid-
ity. Third, the coefficients of determination (r2s) of 0.77,
0.71, and 0.76 indicated that 77%, 71%, and 76% of the var-
iances in G, OS, and B, respectively, could be attributed to
the AVQI and ABI.
ROC analysis of diagnostic accuracy
The AUC of the AVQI was 0.955, suggesting an excellent
predictive power for discriminating between vocally healthy
controls and dysphonic voice. In the Korean population, an
AVQI value of 3.154 was identified as the best cutoff point
between the vocally healthy controls and dysphonia groups.
This value provided the best balance between Sens. and
Spec., with excellent values of 90% and 89%, respectively.
In addition, the LR values at this point were acceptable (LR
+: 8.45 and LR-: 0.12) (Figure 2).
The AUC of the ABI was 0.948, which also suggested a
strong predictive power for discriminating vocally healthy
controls from dysphonic voice. Here, an ABI value of 3.685
was identified as the optimal cutoff for distinguishing

TABLE 2.
Descriptive Outcomes Between the Hand-Marked Selection Number and Standardized Selection Number of 26 Syllables
of Time, AVQI, and ABI Values
Time (in seconds) AVQI ABI
Mean SD Mean SD Mean SD
Hand-marked selection 3.0014 0.0745 3.1206 1.4453 3.6689 1.4842
Standardized selection of 26 syllables 2.9974 0.1140 2.9801 1.5932 3.6042 1.5159
Abbreviations: ABI, acoustic breathiness index ; AVQI, acoustic voice quality index.
 ARTICLE IN PRESS
6 Journal of Voice, Vol. &&, No. &&, 2019

TABLE 4.
Inter-Rater Reliability of the Three SLPs Who Judged G, B, and OS in A-P Ratings
Rater 2 Rater 3
ICC on G ICC on B ICC on OS ICC on G ICC on B ICC on OS
Rater 1 0.879** 0.791** 0.807** 0.777** 0.813** 0.832**
Rater 2 0.867** 0.872** 0.859**
** P < 0.01.
Abbreviations: B, breathiness; G, grade; ICC, intraclass correlation coefficient; OS, overall severity.

FIGURE 1. Comparison of acoustic variables (AVQI and ABI) between normal and pathological voice groups (A: AVQI, B: ABI).

breathy voice, with significant Sens. and Spec. values of 88%
and 86%, respectively. The LR values at this point were
excellent (LR+: 6.47 and LR-: 0.14). Finally, a pairwise
comparison of the ROC curves revealed no significant dif-
ference between the AVQI and ABI (z = 1.353, P = 0.1759)
(Table 6; Figure 3).
DISCUSSION
In this study, we validated the AVQI and ABI tests with a
dataset of voice samples recorded from speakers of Korean
language and investigated the predictive power of these
measures to discriminate dysphonia. Specifically, we evalu-
ated the severity of various voice disorders in our dataset
and identified significant differences between vocally
healthy controls and those with pathological voice disor-
ders. In particular, we identified strong correlations among
both the AVQI and ABI tests with the A-P variables and
calculated the cut-off values that most accurately discrimi-
nate between the two voice groups.
First, we confirmed the number of syllables required to
balance the influences of SV and CS in the Korean language

TABLE 5.
Independent t Test Results for Comparison Between Normal and Pathological Groups in Acoustic Analyses and A-P
Ratings
Group Value (Mean§SD) t P
AVQI Vocally healthy 2.12 § 0.95 -26.0731 0.01**
Pathological 5.53 § 1.81
ABI Vocally healthy 2.56 § 1.06 28.022 0.01**
Pathological 5.64 § 1.49
Grade Vocally healthy 0.00 § 0.00 31.849 0.01**
Pathological 1.49 § 0.66
Breathiness Vocally healthy 0.00 § 0.00 26.121 0.01**
Pathological 1.35 § 0.66
Overall severity Vocally healthy 15.53 § 7.16 37.217 0.01**
Pathological 47.37 § 16.94
** P < 0.01.
Abbreviations: AVQI, acoustic voice quality index version 3.01; ABI, acoustic breathiness index.
 ARTICLE IN PRESS
Geun-Hyo Kim, et al Validation of Acoustic Voice Quality Index Version 3.01 and Acoustic Breathiness 7

FIGURE 2. Correlation between AVQI, ABI and A-P ratings (G, OS and B).A: AVQI and Grade, B: AVQI and Overall Severity, C: ABI
and Breathiness.

and then analyzed additional speech samples using the iden- Second, a comparison of the acoustic analyses and A-P
tified SSN of 26 syllables. This value is similar to the SSNs ratings between the vocally healthy controls and pathologi-
reported for the German16 (27 syllables), Japanese18 (30 syl- cal voice groups revealed statistically significant differences
lables), Dutch25 (34 syllables), and Spanish13 (33 syllables) in all variables. These differences were confirmed by quanti-
languages. This SSN of 26 syllables did not yield statistically fying normal and pathological voices based on multivariate
significant differences with respect to the syllable length, models such as the AVQI and ABI. Although these two
AVQI, and ABI. Furthermore, the AVQI and ABI values
obtained using the SSN of 26 were fairly consistent with
those determined using individual HM syllables. Our results
suggest that this SSN facilitates a highly reliable and bal-
anced estimation of both voice tasks.

TABLE 6.
Pairwise Comparison of ROC Curve
AVQI-ABI
Difference between areas 0.00682
Standard errora 0.00504
95% Confidence interval 0.00306»0.0167
Z statistics 1.353
Significance level P = 0.1759 FIGURE 3. ROC analysis to discriminate normal and pathologi-
cal voice groups
 ARTICLE IN PRESS
8 Journal of Voice, Vol. &&, No. &&, 2019
indexes differed with respect to the included parameters,
both the AVQI and ABI were designed such that larger val-
ues would indicate a poorer voice quality, thus reflecting the
severity of the voice pathology. Consistent with the intended
design, the vocally healthy group in our study exhibited
lower AVQI and ABI values than those in the dysphonia
group. Previous studies similarly reported higher AVQI and
ABI values for subjects with voice disorders of various etiol-
ogies relative to vocally healthy subjects.5,10,16,18,37,38
Third, we identified statistically significant correlations
between the acoustic indexes and various A-P ratings. Both
the AVQI and ABI were strongly correlated with the G, B,
and OS, (rs > 0.84, P < 0.01). In other words, both indexes
appear to form strong correlations with the auditory percep-
tions of overall voice quality and breathiness severity in a
Korean population. In previous studies, relatively high con-
current validity assessments of the A-P ratings with the
AVQI (rs: 0.73−0.92) and ABI (rs: 0.83−0.89) were
reported.5,10,12,14-17,37,39-42 These correlations between the
acoustic and A-P evaluations confirm that the former can
be considered multivariate models associated with the per-
ceived severity of overall dysphonia and breathiness, consis-
tent with previous acoustic studies of these vocal
qualities.26,32,40,42-44 These acoustic evaluations differ from
traditional analysis models, which were used to analyze SV
or CS individually.45-48 In contrast, the AVQI and ABI are
determined using a multistep process involving the (1) con-
catenation of SV and CS segments from samples, (2) extrac-
tion only of voiced segments, and (3) calculation of models
with multiple variables. In addition, our study included
more than 1,600 voice samples from subjects with voice dis-
orders of various types and severity levels, which were ana-
lyzed automatically using the PRAAT script. An important
aspect of the PRAAT script was the nonvoiced extraction,
which has applied the algorithms presented in a previous
study47 and will need to be developed to apply the non-
voiced extraction methods recently introduced to the
PRAAT.49-51 Accordingly, our report presents a multivari-
ate large-group analysis of the acoustic measurements corre-
sponding to overall voice quality and breathiness.
Fourth, both the AVQI and ABI exhibited high levels of
predictive power for discriminating between dysphonic and
normal voices. ROC curve analysis of AVQI identified an
AUC of 0.955, which was comparable to the values identi-
fied previously for the Dutch (AUC = 0.98 and 0.92), Ger-
man (AUC = 0.90), Japanese (AUC = 0.92) and Spanish
languages (AUC = 0.91). Moreover, previous studies have
identified optimal AVQI cut-off values of <2.43
(Sens. = 78.5%, Spec. = 93.2%), <1.85 (Sens. = 72%,
Spec. = 90%), <2.06 (Sens. = 72.1%, Spec. = 93.8%), and
<2.28 (Sens. = 74.8%, Spec. = 94.6%) for the Dutch,8,25 Ger-
man,16 Japanese,12 and Spanish13 languages, respectively.
In our study, an optimal AVQI cut-off value of <3.15 was
determined, with a Sens. of 90% and Spec. of 89%. This
AVQI cutoff value is relatively large, compared to those
determined for other languages. The LR statistics for the
AVQI were also acceptable (LR+: 8.45, LR-: 0.12) and
identified AVQI as a meaningful index. Again, these results
were comparable to those of previous studies of the Dutch
(LR+: 19.9, LR-: 0.27)6, English (LR+: 10.25, LR-: 0.20)52,
German16 (LR+: 7.4, LR-: 0.31), Japanese43 (LR+: 15.1,
LR-: 0.29), and Spanish13 (LR+: 13.8, LR-: 0.27)
languages.
Regarding breathiness severity, the ABI yielded an AUC
of 0.948, indicating a good capacity to distinguish breathi-
ness in the voice signal. This result is comparable to those of
previous studies of the Dutch10 (AUC = 0.95), German16
(AUC = 0.91), Japanese18 (AUC = 0.89) and Spanish13
(AUC = 0.92) languages and thus confirms the high predic-
tive ability of this index for the assessment of breathiness. In
the Korean language, we identified an optimal cut-off of
<3.69 for distinguishing between normal and breathy voi-
ces, with a Sens. of 88% and a Spec. of 86%. The LR statis-
tics were also acceptable, but lower than those reported
from previous studies of the Dutch10 (LR+: 11.6, LR-:
0.19), German16 (LR+: 15.6, LR-: 0.3), Japanese18 (LR+:
8.09, LR-: 0.25), and Spanish13 (LR+: 16.0, LR-: 0.27) lan-
guages. Both AVQI and ABI showed statistically high levels
of discriminative predictive power, with no statistically sig-
nificant differences between the two acoustic variables. This
means that both variables have excellent discriminative
predictability and are useful for discriminating normal voice
from pathological voice. In this study, we conducted AVQI
and ABI measurements for entire voice groups, and we also
need to study groups with specific voice characteristics (e.g.,
dominant in breathy voice). A further study comparing
groups with breathy and nonbreathy voices is expected to
produce interesting results. In addition, we will study the
latest voice extraction algorithm and apply it to PRAAT.
This study has the following limitation: The multivariate
models used in this study are affected by breathy voice. In a
previous pilot study, we found that these models have lim-
ited predictive ability to detect rough voice, strained voice,
tremulous voice, etc. For the evaluation of various voice dis-
orders, future studies should develop multivariate models
that will detect not only the characteristics of breathy voice
but also that of rough voice, strained voice, tremulous voice,
etc.
Future directions
The key point of this study is that regression equations for
AVQI and ABI indexes are being developed. As regression
equations change, the outcome of the study may also
change. The regression equations should be continually
developed to improve the ability to quantify and differenti-
ate pathological voices.
CONCLUSION
In this study, we explored the combined validity of the
AVQI and ABI tests for acoustic analyses and examined the
variables that could significantly distinguish between patho-
logical and normal voice in a cohort of Korean speakers.
We used the methodologies of both indexes to quantify
 ARTICLE IN PRESS
Geun-Hyo Kim, et al Validation of Acoustic Voice Quality Index Version 3.01 and Acoustic Breathiness
normal and dysphonic voices and identified strong correla-
tions of AVQI with A-P ratings G and OS, and of ABI with
A-P rating B. We further identified the optimal cutoff values
of the AVQI and ABI tests that could be used to discrimi-
nate between normal and pathological voices, as well as
evaluated the corresponding diagnostic predictive accura-
cies. Our results demonstrate strong concurrent validity of
the AVQI and ABI tests to quantify the severity of dyspho-
nia, compared to the overall severity and breathiness in our
sample cohort of Korean speakers. Finally, we assessed the
effectiveness of both the AVQI and ABI tests to successfully
quantify and discriminate voice disorders. We believe that
AVQI- and ABI-based analyses of voice samples will facili-
tate the differentiation and evaluation of voice disorders in
clinical settings.
SUPPLEMENTARY MATERIALS
Supplementary material associated with this article can be
found in the online version at https://doi.org/10.1016/j.
jvoice.2019.10.005.
REFERENCES
1. Kreiman J, Gerratt BR, Berke GS. The multidimensional nature of
pathologic vocal quality. J Acoust Soc Am. 1994;96:1291–1302.
2. Barsties B, De Bodt M. Assessment of voice quality: Current state-of-
the-art. Auris Nasus Larynx. 2015;42:183–188.
3. Hirano M. Psyco-acoustic evaluation of voice. Clin Exam Voice: Dis-
orders Human Commun. 1981:81–84.
4. Kempster GB, Gerratt BR, Abbott KV, Barkmeier-Kraemer J, Hill-
man RE. Consensus auditory-perceptual evaluation of voice: develop-
ment of a standardized clinical protocol. Am J Speech-Lang Pat.
2009;18:124–132.
5. Maryn Y, De Bodt M, Roy N. The Acoustic Voice Quality Index:
Toward improved treatment outcomes assessment in voice disorders. J
Commun Disord. 2010;43:161–174.
6. Maryn Y, Corthals P, Van Cauwenberge P, Roy N, De Bodt M.
Toward Improved ecological validity in the acoustic measurement of
overall voice quality: combining continuous speech and sustained vow-
els. J Voice. 2010;24:540–555.
7. Maryn Y, Weenink D. Objective dysphonia measures in the program
Praat: smoothed cepstral peak prominence and acoustic voice quality
index. J Voice. 2015;29:35–43.
8. Barsties B, Maryn Y. External validation of the acoustic voice quality
index version 03.01 with extended representativity. Ann Otol Rhinol
Laryngol. 2016;125:571–583.
9. Wuyts FL, De Bodt MS, Molenberghs G, et al. The Dysphonia
Severity Index: an objective measure of vocal quality based on a multi-
parameter approach. J Speech Language Hearing Res. 2000;43:796–
809.
10. Barsties v, Latoszek B, Maryn Y, Gerrits E, De Bodt M. The Acoustic
Breathiness Index (ABI): a multivariate acoustic model for breathiness.
J Voice. 2017;31:511–e511.
11. Fritzell B, Hammarberg B, Gauffin J, Karlsson I, Sundberg J.
Breathiness and insufficient vocal fold closure. J Phonetics.
1986;14:549–553.
12. Hosokawa K, Barsties VLB, Iwahashi T, et al. The Acoustic voice
quality index version 03.01 for the Japanese-speaking population.
J Voice. 2019;33:125 e121-125 e112.
13. Hernandez JD, Gomez NML, Jimenez A, Izquierdo LM, vander
Latoszek B. Validation of the acoustic voice quality index version
03.01 and the acoustic breathiness index in the Spanish language. Ann
Otol Rhinol Laryngol. 2018;127:317–326.
9
14. Kim GH, Lee YW, Bae IH, Park HJ, Lee BJ, Kwon SB. Comparison
of two versions of the acoustic voice quality index for quantification of
dysphonia severity. J Voice. 2018.
15. Pommee T, Maryn Y, Finck C, Morsomme D. The Acoustic voice
quality index, version 03.01, in French and the voice handicap index.
J Voice. 2018.
16. Barsties B, Lehnert B, Janotte B. Validation of the acoustic voice qual-
ity index version 03.01 and acoustic breathiness index in German.
J Voice. 2018.
17. Englert M, Lima L, Behlau M. Acoustic voice quality index and acous-
tic breathiness index: Analysis with different speech material in the
Brazilian portuguese. J Voice. 2019.
18. Hosokawa K, von Latoszek BB, Ferrer-Riesgo CA et al. Acoustic
breathiness index for the japanese-speaking population: Validation
study and exploration of affecting factors. J Speech Lang Hear Res
2019:1−15.
19. Ha S, Johnson CJ, Kuehn DP. Characteristics of Korean phonology:
review, tutorial, and case studies of Korean children speaking English.
J Commun Disord. 2009;42:163–179.
20. Teahan WJ, Cleary JG. The entropy of English using PPM-based
models. In: Proceedings of Data Compression Conference-DCC'96.
IEEE; 1996:53–62.
21. Koo HS. A study of production difficulties of English bilabial stops
and labiodental fricatives by Korean learners of English. Phonetics
Speech Sci. 2009;1:11–15.
22. Dejonckere PH, Bradley P, Clemente P, et al. A basic protocol for
functional assessment of voice pathology, especially for investigating
the efficacy of (phonosurgical) treatments and evaluating new assess-
ment techniques − guideline elaborated by the committee on phoniat-
rics of the European Laryngological Society (ELS). Eur Arch Oto-
Rhino-L. 2001;258:77–82.
23. Ingrisano DRS, Perry CK, Jepson KR. Environmental noise: a threat
to automatic voice analysis. Am J Speech-Lang Pat. 1998;7:91–96.
24. Deliyski DD, Shaw HS, Evans MK. Adverse effects of environmental
noise on acoustic voice quality measurements. J Voice. 2005;19:15–28.
25. Barsties B, Maryn Y. The improvement of internal consistency of the
Acoustic Voice Quality Index. Am J Otolaryngol. 2015;36:647–656.
26. Kim GH, Lee YW, Bae IH, Park HJ, Wang SG, Kwon SB. Validation
of the acoustic voice quality index in the Korean language. J Voice.
2018.
27. Portney LG. Foundations of Clinical Research: Applications to Prac-
tice. 3th (third) Edition 2009.
28. Hallgren KA. Computing inter-rater reliability for observational data:
an overview and tutorial. Tutorials Quantitative Methods Psychol.
2012;8:23.
29. Webb A, Carding P, Deary IJ, MacKenzie K, Steen N, Wilson JA.
The reliability of three perceptual evaluation scales for dysphonia.
Euro Arch Oto-Rhino-Laryngol Head Neck. 2004;261:429–434.
30. Yiu EM, Ng CY. Equal appearing interval and visual analogue scaling
of perceptual roughness and breathiness. Clin Linguist Phon.
2004;18:211–229.
31. Mutlu A, Livanelioglu A, Gunel MK. Reliability of Ashworth and
Modified Ashworth scales in children with spastic cerebral palsy. BMC
Musculoskelet Disord. 2008;9:44.
32. Maryn Y, Kim HT, Kim J. Auditory-Perceptual and Acoustic meth-
ods in measuring dysphonia severity of Korean speech. J Voice.
2016;30:587–594.
33. McGraw KO, Wong SP. Forming inferences about some intraclass
correlation coefficients. Psychol Methods. 1996;1:30–46.
34. Mukaka MM. A guide to appropriate use of correlation coefficient in
medical research. Malawi Med J. 2012;24:69–71.
35. Delong ER, Delong DM, Clarkepearson DI. Comparing the areas
under 2 or more correlated receiver operating characteristic curves - a
nonparametric approach. Biometrics. 1988;44:837–845.
36. Muller MP, Tomlinson G, Marrie TJ, et al. Can routine laboratory
tests discriminate between severe acute respiratory syndrome and other
causes of community-acquired pneumonia? Clin Infect Dis.
2005;40:1079–1086.
 ARTICLE IN PRESS
10
37. Englert M, Lima L, Constantini AC, Latoszek BBV, Maryn Y, Behlau
M. Acoustic Voice Quality Index - AVQI for Brazilian Portuguese
speakers: analysis of different speech material. Codas. 2019;31:
e20180082.
38. Barsties VLB, Ulozaite-Staniene N, Petrauskas T, Uloza V, Maryn Y.
Diagnostic accuracy of dysphonia classification of DSI and AVQI.
Laryngoscope. 2019;129:692–698.
39. Pommee T, Maryn Y, Finck C, Morsomme D. Validation of the
acoustic voice quality index, version 03.01, in French. J Voice. 2018.
40. Barsties VLB, Ulozaite-Staniene N, Maryn Y, Petrauskas T, Uloza V.
The influence of gender and age on the acoustic voice quality index
and Dysphonia severity index: a normative study. J Voice.
2019;33:340–345.
41. Uloza V, Barsties B, Ulozaite-Staniene N, Petrauskas T, Maryn Y. A
comparison of Dysphonia Severity Index and Acoustic Voice Quality
Index measures in differentiating normal and dysphonic voices. Eur
Arch Otorhinolaryngol. 2018;275:949–958.
42. Uloza V, Petrauskas T, Padervinskis E, Ulozaite N, Barsties B, Maryn
Y. Validation of the acoustic voice quality index in the Lithuanian lan-
guage. J Voice. 2017;31:257–e251.
43. Hosokawa K, Barsties B, Iwahashi T, et al. Validation of the acoustic
voice quality index in the Japanese language. J Voice. 2017;31:260–
e261.
44. N
un

~ez-Batalla F, Díaz-Fresno E, Alvarez-Fern
andez A, Mu~ noz Cor-
dero G, Llorente Pend
as JL. Application of the acoustic voice quality
Journal of Voice, Vol. &&, No. &&, 2019
index for objective measurement of Dysphonia severity. Acta Otorrino-
laringol (English Edition). 2017;68:204–211.
45. Watts CR, Awan SN, Worth F, Bodt D, Cauwenberge V. Use of spec-
tral / cepstral analyses for differentiating normal from hypofunctional
voices in sustained vowel. J Speech, Language Hearing Res.
2011;54:1525–1538.
46. Awan SN, Roy N, Dromey C. Estimating dysphonia severity in con-
tinuous speech: application of a multi-parameter spectral/cepstral
model. Clin Linguist Phon. 2009;23:825–841.
47. Parsa V, Jamieson DG. Acoustic discrimination of pathological voice:
sustained vowels versus continuous speech. J Speech Lang Hear Res.
2001;44:327–339.
48. Hillenbrand J, Houde RA. Acoustic correlates of breathy vocal qual-
ity: dysphonic voices and continuous speech. J Speech Lang Hear Res.
1996;39:311–321.
49. Kumar SBS, Rao KS. Voice/non-voice detection using phase of zero
frequency filtered speech signal. Speech Commun. 2016;81:90–103.
50. Germain FG, Sun DL, Mysore GJ. Speaker and noise independent
voice activity detection. in. Proc. Interspeech. 2013:732–736.
51. Gorriz JM, Ramirez J, Lang EW, Puntonet CG, Turias I. Improved
likelihood ratio test based voice activity detector applied to speech rec-
ognition. Speech Commun. 2010;52:664–677.
52. Reynolds V, Buckland A, Bailey J, et al. Objective assessment of pedi-
atric voice disorders with the acoustic voice quality index. J Voice.
2012;26:672–e671.