Improving Test-Retest Reliability of Quantitative
Electroencephalography Using Different Preprocessing Approaches*
Jazmin Suarez-Revelo, John Ochoa-Gomez, Jon Duque-Grajales

Abstract— This work aims to assess the effect of
preprocessing approaches over test–retest reliability of
quantitative electroencephalography measurements. Two
electroencephalography sessions were recorded during an eyes-
closed resting state condition in 15 young healthy individuals.
The second session was 4 to 6 weeks after the first one. Clean
recordings were obtained from the implementation of different
preprocessing approaches commonly used in the literature. We
then estimated the power spectrum density, for each individual
and preprocessing approach, in six frequency bands: delta,
theta, alpha1, alpha2, beta, and gamma. Test-retest reliability
using the intraclass correlation coefficient was calculated for
power spectrum in each methodology and frequency band. We
found that the test-retest reliability varied considerably across
frequency bands and preprocessing approaches. Reliability was
higher for theta, alpha1, and alpha2 frequency bands. Also, the
use of preprocessing approach that includes a robust reference
to average and independent component analysis, can improve
test-retest reliability in other bands such as beta and gamma.
Results suggest that quantitative electroencephalography are
test-retest reliable and can be used in longitudinal studies.
I. INTRODUCTION
The electroencephalography (EEG) is a technique used in
brain mapping with applications in research and clinical
practice [1]. It provides a noninvasiveness representation of
electrophysiological brain activity, reflecting both local and
global activity with high temporal resolution. The different
rhythms measured by EEG reflect the current state of the
brain and help to identify dysfunction related to disease as
epilepsy, structural abnormalities, and cognitive impairment
[2]. Quantitative EEG (qEEG) comprises a set of statistical
and signal processing methods for the analysis of frequency
and amplitude of EEG signals. It enables the detection of
abnormal patterns associated with specific pathological
conditions and regulatory patterns that exist in cognitive
conditions [3]. Previous electrophysiological studies have
shown changes in large-scale patterns of synchronized
neuronal activity and a considerable variability over time [4].
EEG analysis has been based mainly on statistical processing
to obtain stable and reliable characteristics because brain
activity is essentially nonstationary, nonlinear, and noisy [5],
[6]. In order to characterize disease states and progression,
qEEG measurements should have low inter-subject
variability and high test-retest reliability.
* Research supported by Universidad de Antioquia under the project
“Identificación de marcadores preclínicos de la mutación E280A de la
enfermedad de Alzheimer a partir de medidas de conectividad en EEG”
identified by the code PRG14-1-02F.
J Suárez, J. Ochoa, J. Duque are with Bioinstrumentation and Clinical
Research Group, Bioengineering Program, Universidad de Antioquia UdeA,
Calle 70 # 52-51, Medellin, Colombia (e-mail: jazmin.suarez@udea.edu.co,
john.ochoa@udea.edu.co, jon.duque@udea.edu.co, respectively).
978-1-4577-0220-4/16/$31.00 ©2016 IEEE
A critical step in the EEG analysis is the data
preprocessing which increase the signal-to-noise ratio and
remove unwanted artifacts in the data. Common EEG
preprocessing approaches include data filtering, interpolation
of bad channels, epoch segmentation, re-referencing data, and
removing of physiological artifacts as eye blinking or muscle
activity. The order and implementation of these steps affect
the quality of signal for further analysis. For example, the
data referencing to average could contaminate the entire
dataset if there are unusual channels [7]; the suppression of
artifacts by independent component analysis (ICA) can lead
to loss of brain signal [8]. In order to validate a protocol for
EEG recording and a preprocessing pipeline that can be
considered in a clinical follow-up, this study assess the test-
retest reliability of qEEG measurements calculated in EEG
recordings with different preprocessing methodologies. To
achieve this, we recorded EEG during eyes-closed resting in
15 healthy volunteers in two sessions, each session with four
to six weeks of time difference. The recordings were cleaned
using different EEG preprocessing pipelines commonly
applied [7]–[10]. Power spectrum was estimated in six
frequency bands: delta (1–4 Hz), theta (4–8Hz), alpha1 (8–
10Hz), alpha2 (10–13Hz), beta (13–30Hz), and gamma (30–
50Hz). The intraclass correlation (ICC) was used for measure
the test-retest reliability of qEEG values.
II. METHODOLOGY
A. Subjects and EEG acquisition
EEG signals were recorded twice in 15 young healthy
subjects (23.33 ± 3.02 years) at awake resting (eyes closed
during 5 minutes). The time between sessions was around 4.8
± 0.7 weeks. A Neuroscan unit amplifier (Neuroscan Medical
System, Neurosoft Inc. Sterling, VA, USA) was used to
record EEG signals. Data were recorded (0.05-200 Hz
bandpass) and sampled at 1000 Hz from 58 electrodes
positioned according to the international 10-20 system with
right earlobe reference. Informed consent for participation
was obtained from all subjects according to the protocol
approved by the Human Subjects Committee of the
Universidad de Antioquia.
B. EEG data preprocessing pipelines
We implemented four parallel automated preprocessing
pipelines using two MATLAB toolbox: EEGLAB [11] and
the standardized early-stage EEG processing pipeline (PREP)
[7]. The preprocessing approaches are described below
(Figure 1):
Pipeline 1 includes importing data; application of high-
pass FIR filter at 1 Hz and low-pass FIR filter at 50 Hz;
detection of noisy channels based on kurtosis, probability,
and spectrum measures; interpolation of these channels using
spherical splines; epoching data to 2s; re-referencing the data
961
to average; removing epochs with ocular, muscular, and other III. RESULTS
types of artifact by a procedure based on linear trend, joint
probability and kurtosis approach [9]; and independent Table I shows the number of channels interpolated,
component analysis enhanced by wavelet (wICA) to correct epochs removed, and epochs remaining in each preprocessing
remaining eye blinks artifacts [8]. wICA uses a wavelet pipeline. Figure 2 shows a comparison of applying classical
thresholding to the demixed independent components as an data average reference (pipeline 1) and using the robust
intermediate step. This step enables recovering the neural average reference (other pipelines). Figure 2a show DC
activity that exists in artefactual components that are detected shifting levels in the pipeline 1 that are corrected with the
using the Multiple Artifact Rejection Algorithm [12]. application of robust reference (Figure 2b). Figure 3 shows
Pipeline 2 comprises importing data; PREP pipeline, which the effect of applying analysis wICA. In Figure 3a there is
includes a robust reference to average, where bad channels noise produced by ocular activity which is corrected with the
are excluded, and detection and interpolation of bad channels application of wICA (Figure 3b).
relative to this reference [7]; filtering data using high pass
TABLE I. RESULTING NUMBER OF EPOCHS AND INTERPOLATED
FIR filter (1 Hz) and low-pass FIR filter (50 Hz); and epoch CHANNELS.
segmentation. Pipeline 3 includes the pipeline 2 followed by
removing epochs step used in pipeline1. Pipeline 4 comprises Number of
Number of Number of
interpolated
the pipeline 2 followed by wICA analysis. Prepro channels
removed epochs remaining epochs
cessing Mean ± SD Mean ± SD
Figure 1 Summary of EEG preprocessing pipelines. Mean ± SD
S1 S2 S1 S2 S1 S2
1.7 1.9 65.5 109.3 64.8 46.3
P1
±1 ± 0.7 ± 39 ± 15.5 ± 27.9 ± 15
6.1 7.4 130.3 155.7
P2 - -
± 4.7 ±3 ± 29.6 ± 4.8
6.1 7.4 13.9 ± 19.4 116.4 136.3
P3
± 4.7 ±3 6.4 ± 6.4 ± 24.1 ± 6.8
6.1 7.4 130.3 155.7
P4 - -
± 4.7 ±3 ± 29.6 ± 4.8
P1: Pipeline 1. P2: Pipeline 2. P3: Pipeline 3. P4: Pipeline 4. S1: session 1. S2: session 2. SD:
standard deviation.

Figure 2 Comparison of traditional and robust average references. a)

Classical average referencing. b) Robust average referencing. (Red
rectangles show segments corrected. The data has been band-pass filtered
at 1-50 Hz).
C. qEEG analysis
Power spectrum density was computed using the Welch
approach [13] in 30 artifact-free epochs. These epochs were
randomly selected from each recording. Relative power
spectrum was calculated as the ratio of the power in a band
over total power spectrum. The power in six frequency bands
was calculated: delta (1–4 Hz), theta (4–8Hz), alpha1 (8–
10Hz), alpha2 (10–13Hz), beta (13–30Hz), and gamma (30–
50Hz). As result an array of 58x6x30 with power values for
each subject and preprocessing strategy was obtained. In the
array 58 is the number of channels, 6 the number of bands,
and 30 the number of epochs. The original 58x6x30 matrix
was reduced to 58x6 by the power averaging across the 30
epochs.
D. Statistical analysis
Intraclass correlation coefficient (ICC) derived from a
two way mixed model with an absolute agreement definition
was calculated between the two EEG recording sessions to
quantify the test-retest reliability of qEEG measurements The mean ICC values calculated over all channels for
[14]. ICCs are reported descriptively and then we used a two- each frequency band and processing pipeline are shown in
way analysis of variance (ANOVA) model to test hypotheses Figure 4. The mean ICC ranged from 0.38 to 0.8 with a mean
about effects of frequency band and preprocessing approach of 0.62 ± 0.14 (SD). Test-retest reliability varied considerably
on ICCs. Statistical significance of 5% was used for all tests. over different frequency bands. The lowest mean ICC was
All statistical comparisons and validation of assumptions found in the delta (0.47 ± 0.07 (SD)) and gamma band (0.48
were implemented in MATLAB. ± 0.05 (SD)), and the highest was found in the alpha2 band
(0.79 ± 0.02 (SD)). The ANOVA analysis showed interaction
effect among the factors Pipeline and Band (F(15)=3.28,

962
P=2.11e-5), which indicated that preprocessing approach was
not associated with increasing test-retest reliability in all
frequency bands. Figure 5 shows an interaction plot among
pipeline and frequency band; ICC values decrease in delta,
theta, alpha1, and beta bands in the pipeline 1. It is also noted
that with the pipeline 4 ICC values are less dispersed around
the six frequency bands, and it improves test-retest reliability
in the beta and gamma bands.
Figure 3. Effect of wICA in the signal. a) Without wICA (pipeline 2); b)
With wICA (pipeline 4). (Red rectangles show segments corrected).
Figure 6 shows ICC mean values for each electrode over
a surface rendering of the scalp. For alpha1 and alpha2 bands
test-retest reliability is higher and uniformly distributed
throughout the scalp for all preprocessing pipelines. In theta
band is observed that in central region reliability is lower, but
this increases with the application of pipeline 4. Similar
characteristics are observed for delta, beta and gamma bands,
where the distribution is less uniform and the ICC values
lower for pipeline 1.
Figure 6. Scalp map representation of test-retest reliability (measure by
ICC) of qEEG measures. Results for the six frequency bands are given from
top to bottom and are categorized according to preprocessing pipeline from
left to right.

Figure 4. Mean ICC values. (δ: delta, Ө: theta, α1: alpha1, α2: alpha2, β:
beta, γ: gamma. ICC: intraclass correlation coefficient. Vertical bars show
the standard deviation.)
IV. DISCUSSION
In this work we studied the test-retest reliability of qEEG
measurements at resting state between two recording
sessions. We found that in general qEEG measures showed
good test-retest reliability. However, this reliability varied
considerably through frequency bands and preprocessing
approaches, being higher for alpha2, theta, and alpha1 bands.
Additionally, it was found that the use of a suitable
processing pipeline, which includes a robust reference to
average and wICA analysis, could improve the test-retest
reliability in other frequency bands such as beta and gamma.
The use of PREP toolbox for referencing data and
interpolation of bad channels improved visual quality of
Figure 5. Interaction plot. (Band 1: delta, 2: theta, 3: alpha1, 4: alpha2, 5: signals and allowed to retain more data for analysis. The
beta, 6: gamma)
Pipeline robust reference correct the DC shifting because this
1 2 3 4
0.8
algorithm does not consider bad channels to calculate
Band 0.75
reference and the average is not affected by outliers.
Preprocessing approaches that included the use of PREP
0.7
Band = 1 0.65
Band = 2
(pipelines 2, 3, and 4) interpolated more channels than the
ICC

0.6
Band = 3
0.55
Band = 4
Band = 5 0.5 one who does not used it (pipeline 1), because this technique
Band = 6 0.45
0.4
includes more rigorous selection criteria for detection of bad
0.8
channels. These criteria include extreme amplitudes, lack of
0.75 Pipeline correlation with any other channel, lack of predictability by
0.7
0.65
other channels, and unusual high frequency noise [7]. Due to
Pipeline = 1
bad channel interpolation and robust reference to average is
ICC

0.6
Pipeline = 2

also noted that the number of epochs removed is less for

0.55
Pipeline = 3
0.5 Pipeline = 4
0.45
0.4
pipeline 3 than pipeline 1. It is important to note wICA
1 2 3 4 5 6
analysis corrects eye blinks artifacts. This analysis is based
Band on ICA signal decomposition and includes as an intermediate
963
step the wavelet thresholding of the independent components
[8]. This step recovers the low amplitude and broad band
neural activity persistent in the components identified as
responsible for artifacts. An advantage of wICA is its
automation so laborious visual inspection of the independent
components is not required. Instead, all components are
passed through the thresholding procedure and only the high
magnitude artifacts are cut out.
Several studies have reported test-retest reliability of
spectral power during resting state. Particularly, Cannon et al
[15] reported reproducibility of qEEG measurements at 30
days, obtaining good reproducibility for absolute power and
coherence in resting state with an ICC average of 0.81 for
closed eyes. Evaluating test-retest reliability at rest, but in
other time intervals, other studies have reported good
reliability in alpha and theta bands with eyes closed [16],
[17]. Our results confirm previously reported indicating
higher test-retest reliability for theta, alpha1 and alpha2
bands, and showing how it can improve with the use of
proposed techniques for signal preprocessing. In previous
studies the reliability of power spectrum was calculated in
specific channels. Here, we assess test-retest reliability over
whole scalp providing a more complete analysis.
Furthermore, unlike previous works, we develop an
automatic signal preprocessing pipeline which makes easy
the analysis and exclude subjectivity in epoch or components
removal. The EEG recording of eyes-closed resting state
represents a fully standardized procedure very easy and rapid
to be carried out in clinical environment. The use of resting
state EEG does not require stimulation devices or registration
of subject’s behavior, and is not prone to fatigue and anxiety
typically associated to task performance. It is more suitable
when EEG recordings are performed in elderly subjects.
Furthermore, resting state cortical EEG rhythms can be
recorded in highly comparable experimental conditions in
normal subjects, individuals with subjective memory
complaints, objective mild objective cognitive impairment,
and even Alzheimer’s disease [18].
V. CONCLUSION
The acquisition of reliable electrical brain activity is a
topic of great interest because their importance in the
development of diagnostic biomarkers and as mechanism to
evaluate treatment results. Our results confirm the test-retest
reliability of qEEG measures under resting condition in
theta, alpha1 and alpha2 bands and open the possibility of
using these information in longitudinal studies. Future
studies will include evaluation of other experimental factors
like acquisition under a task, and other analysis like
functional connectivity and graph theory. These result also
allow the application of reliable EEG indices in the
evaluation of the Alzheimer’s disease evolution.
ACKNOWLEDGMENT
This work was supported by Vicerrectoría de
Investigación of Universidad de Antioquia (CODI), Project
“Identificación de marcadores preclínicos de la mutación
E280A de la enfermedad de Alzheimer a partir de medidas
de conectividad en EEG”, code PRG14-1-02.
964
REFERENCES
[1] D. Millett, P. Coutin-Churchman, and J. M. Stern, “Basic
Principles of Electroencephalography,” in Brain Mapping,
Elsevier, 2015, pp. 75–80.
[2] D. B. MacDonald, Electroencephalography: Basic Principles and
Applications. Elsevier, 2015, pp. 353–363.
[3] A. P. Medeiros, R. Anghinah, M. T. Smidth, and J. M. Silva, “The
clinical use of quantitative EEG in cognitive disorders,” Dement
Neuropsychol, vol. 3, no. 3, pp. 195–203, 2009.
[4] W. Klonowski, “Everything you wanted to ask about EEG but
were afraid to get the right answer.,” Nonlinear Biomed. Phys.,
vol. 3, p. 2, 2009.
[5] P. Indic, R. Pratap, V. P. Nampoori, and N. Pradhan,
“Significance of time scales in nonlinear dynamical analysis of
electroencephalogram signals.,” Int. J. Neurosci., vol. 99, no. 1–4,
pp. 181–194, Aug. 1999.
[6] A. Y. Kaplan, A. a. Fingelkurts, A. a. Fingelkurts, S. V. Borisov,
and B. S. Darkhovsky, “Nonstationary nature of the brain activity
as revealed by EEG/MEG: Methodological, practical and
conceptual challenges,” Signal Processing, vol. 85, pp. 2190–
2212, 2005.
[7] N. Bigdely-Shamlo, T. Mullen, C. Kothe, K.-M. Su, and K. a.
Robbins, “The PREP pipeline: standardized preprocessing for
large-scale EEG analysis,” Front. Neuroinform., vol. 9, no. June,
pp. 1–20, 2015.
[8] N. P. Castellanos and V. a. Makarov, “Recovering EEG brain
signals: Artifact suppression with wavelet enhanced independent
component analysis,” J. Neurosci. Methods, vol. 158, pp. 300–
312, 2006.
[9] A. Delorme, T. J. Sejnowski, and S. Makeig, “Enhanced detection
of artifacts in EEG data using higher-order statistics and
independent component analysis,” Neuroimaging, vol. 34, no. 4,
pp. 1443–1449, 2007.
[10] A. Delorme, T. Mullen, C. Kothe, Z. Akalin Acar, N. Bigdely-
Shamlo, A. Vankov, and S. Makeig, “EEGLAB, SIFT, NFT,
BCILAB, and ERICA: New tools for advanced EEG processing,”
Comput. Intell. Neurosci., vol. 2011, 2011.
[11] A. Delorme and S. Makeig, “EEGLAB: An open source toolbox
for analysis of single-trial EEG dynamics including independent
component analysis,” J. Neurosci. Methods, vol. 134, pp. 9–21,
2004.
[12] I. Winkler, S. Haufe, and M. Tangermann, “Automatic
Classification of Artifactual ICA-Components for Artifact
Removal in EEG Signals,” Behav. Brain Funct., vol. 7, no. 1, p.
30, 2011.
[13] P. Welch, “The use of fast Fourier transform for the estimation of
power spectra: A method based on time averaging over short,
modified periodograms,” IEEE Trans. Audio Electroacoust., vol.
15, no. 2, pp. 70–73, 1967.
[14] K. O. McGraw and S. P. Wong, “‘Forming inferences about some
intraclass correlations coefficients’: Correction.,” Psychol.
Methods, vol. 1, no. 1, pp. 390–390, 1996.
[15] R. L. Cannon, D. R. Baldwin, T. L. Shaw, D. J. Diloreto, S. M.
Phillips, A. M. Scruggs, and T. C. Riehl, “Reliability of
quantitative EEG (qEEG) measures and LORETA current source
density at 30 days,” Neurosci. Lett., vol. 518, no. 1, pp. 27–31,
2012.
[16] L. . McEvoy, M. . Smith, and A. Gevins, “Test–retest reliability
of cognitive EEG,” Clin. Neurophysiol., vol. 111, no. 3, pp. 457–
463, Mar. 2000.
[17] M. Corsi-Cabrera, L. Galindo-Vilchis, Y. del-Río-Portilla, C.
Arce, and J. Ramos-Loyo, “Within-subject reliability and inter-
session stability of EEG power and coherent activity in women
evaluated monthly over nine months,” Clin. Neurophysiol., vol.
118, pp. 9–21, 2007.
[18] F. Vecchio, C. Babiloni, R. Lizio, F. De Vico Fallani, K.
Blinowska, G. Verrienti, G. Frisoni, and P. M. Rossini, “Resting
state cortical EEG rhythms in Alzheimer’s disease: Toward EEG
markers for clinical applications: A review,” Suppl. Clin.
Neurophysiol., vol. 62, pp. 223–236, 2013.