See

discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/11665848

Entropy, entropy rate, and pattern classification

as tools to typify complexity in short heart period
variability series
ARTICLE in IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING DECEMBER 2001
Impact Factor: 2.35 DOI: 10.1109/10.959324 Source: PubMed

CITATIONS

READS

173

48

7 AUTHORS, INCLUDING:
Alberto Porta

Nicola Montano

University of Milan

University of Milan

334 PUBLICATIONS 7,180 CITATIONS

256 PUBLICATIONS 6,291 CITATIONS

SEE PROFILE

SEE PROFILE

Raffaello Furlan

Sergio Cerutti

University of Milan

Politecnico di Milano

176 PUBLICATIONS 7,291 CITATIONS

607 PUBLICATIONS 12,734 CITATIONS

SEE PROFILE

All in-text references underlined in blue are linked to publications on ResearchGate,

letting you access and read them immediately.

SEE PROFILE

Available from: Alberto Porta

Retrieved on: 26 February 2016

1282

IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 48, NO. 11, NOVEMBER 2001

Entropy, Entropy Rate, and Pattern Classification as

Tools to Typify Complexity in Short Heart Period
Variability Series
Alberto Porta*, Stefano Guzzetti, Nicola Montano, Raffaello Furlan, Massimo Pagani, Alberto Malliani, and
Sergio Cerutti, Senior Member, IEEE

AbstractAn integrated approach to the complexity analysis of

short heart period variability series ( 300 cardiac beats) is proposed and applied to healthy subjects during the sympathetic activation induced by head-up tilt and during the driving action produced by controlled respiration (10, 15, and 20 breaths/min, CR10,
CR15, and CR20 respectively). The approach relies on: 1) the calculation of Shannon entropy (SE) of the distribution of patterns
lasting three beats; 2) the calculation of a regularity index based
on an entropy rate (i.e., the conditional entropy); 3) the classification of frequent deterministic patterns (FDPs) lasting three beats.
A redundancy reduction criterion is proposed to group FDPs in
four categories according to the number and type or of heart period
changes: a) no variation (0V); b) one variation (1V); and c) two like
variations (2LV); 4) two unlike variations (2UV). We found that:
1) the SE decreased during tilt due to the increased percentage of
missing patterns; 2) the regularity index increased during tilt and
CR10 as patterns followed each other according to a more repetitive scheme; and 3) during CR10, SE and regularity index were not
redundant as the regularity index significantly decreased while SE
remained unchanged. Concerning pattern analysis we found that:
a) at rest mainly three classes (0V, 1V, and 2LV) were detected; b)
0V patterns were more likely during tilt; c) 1V and 2LV patterns
were more frequent during CR10; and d) 2UV patterns were more
likely during CR20. The proposed approach based on quantification of complexity allows a full characterization of heart period
dynamics and the identification of experimental conditions known
to differently perturb cardiovascular regulation.
Index TermsAutonomic control, conditional entropy, corrected conditional entropy, heart rate variability, pattern analysis,
regularity index, Shannon entropy, symbolic dynamics.
Manuscript received January 10, 2001; revised July 05, 2001. Asterisk indicates corresponding author.
*A. Porta is with the Dipartimento di Scienze Precliniche, Laboratorio Interdisciplinare Tecnologie Avanzate LITA di Vialba, Universita degli Studi di
Milano, LITA di Vialba, Via G.B. Grassi 74, 20157 Milano, Italy (e-mail alberto.porta@unimi.it).
S. Guzzetti and R. Furlan are with the Centro Ricerche Cardiovascolari
CNR, Medicina Interna II, Ospedale L. Sacco, Universita degli Studi di
Milano, 20157 Milan, Italy.
N. Montano is with the Dipartimento di Scienze Precliniche, Laboratorio Interdisciplinare Tecnologie Avanzate LITA di Vialba, Universita degli Studi di
Milano, LITA di Vialba, 20157 Milano.
M. Pagani is with the Dipartimento di Scienze Precliniche, Universita degli
Studi di Milano, LITA di Vialba, Milan, Italy and Medicina Interna I, Ospedale
L. Sacco, Universita degli Studi di Milano, 20157 Milan, Italy.
A. Malliani is with the Dipartimento di Scienze Precliniche, Laboratorio Interdisciplinare Tecnologie Avanzate LITA di Vialba, Universita degli Studi di
Milano, LITA di Vialba, 20157 Milano and also with the Centro Ricerche Cardiovascolari CNR, Medicina Interna II, Ospedale L. Sacco, Universita degli
Studi di Milano, 20157 Milan, Italy.
S. Cerutti is with the Dipartimento di Bioingegneria, Politecnico di Milano,
20157 Milan, Italy.
Publisher Item Identifier S 0018-9294(01)09141-8.

I. INTRODUCTION

HERE is an increasing interest in evaluating complexity

of the short term cardiovascular control via heart period
variability analysis [1][6]. Heart period variability is the result of the activity of vasomotor and respiratory centers [7],
[8], of baroreflex and chemoreflex closed loop regulation [9],
[10] of cardiovascular reflexes mediated by vagal and sympathetic afferences [11], and of vascular autoregulation [12]. All
these mechanisms act over similar but not coincident frequencies (below 0.5 Hz) contributing to the complexity of the signal.
The increased strength of the links among these mechanisms
(usually weakly interacting) or the dominant action of one of
these mechanisms taking priority over the others determines the
reduction of complexity (i.e., the increase of regularity and predictability) and this is considered a marker of pathology [13].
Complexity analysis can be performed through the evaluation of entropy and entropy rate. Entropy [e.g., Shannon entropy
(SE)] calculates the degree of complexity of the distribution of
the samples of a signal. The largest entropy is obtained when
the distribution is flat (the samples are identically distributed).
On the contrary, if some values are more likely (e.g., the sample
distribution is Gaussian), the entropy decreases. Usually, in the
analysis of heart period variability, entropy is not calculated directly over the samples of the series but over patterns of length
(i.e., ordered sequences of samples). In this case, entropy
measures the complexity of the pattern distribution as a function of . Voss et al. [14] found a decrease of the SE of patterns lasting three beats in patients after myocardial infarction
and correlated this decrease to the risk of malignant arrhythmias
and sudden cardiac death. Cysarz et al. [15] found an inverse relationship between the SE of binary patterns and the mean RR
interval. Differently, entropy rate (e.g., approximated entropy or
conditional entropy) provides a global index of complexity of
previous
the distributions of the samples conditioned to
2 was
samples as a function of . Approximated entropy at
found useful to detect infants at risk of sudden infant death syndrome [16]. Conditional entropy (CE) was utilized by Porta et
al. [17] to evaluate the decrease of complexity during the sympathetic activation produced by tilt and parasympathetic blockade
induced by atropine.
Usually, in heart period variability analysis entropy and entropy rates have not been utilized in association [14][17] even
though they provide different information. Indeed, entropy depends on whether there are some patterns more present than

00189294/01$10.00 2001 IEEE

PORTA et al.: TOOLS TO TYPIFY COMPLEXITY IN SHORT HEART PERIOD VARIABILITY SERIES

others, but it does not provide any information about the dynamical relationship among patterns (i.e., the rule linking a pattern to the next one), while entropy rate evaluates whether there
is a repetitive sequence of patterns, thus quantifying regularity
of the signal [18], [19]. Moreover, as complexity indexes do
not give any indication about the pattern type, the classification
tools might provide the complementary information by labeling
the patterns, the frequency, and the recurrency of which are responsible for generating complexity.
The aim of this study is to propose an integrated approach to
the analysis of complexity of heart period variability. Therefore,
an entropy (i.e., SE), an entropy rate (i.e., CE) and a method of
pattern classification are jointly utilized. Entropy and entropy
rate indexes are compared with verify whether they provide non
redundant information even over short data series, while deterministic patterns are classified to find out whether privileged
categories do exit and depend on experimental conditions.
The analysis was performed on data derived from healthy
human subjects during two experimental conditions known to
perturb cardiovascular regulation by inducing a sympathetic activation (head-up tilt) and by forcing heart period variability to
oscillate at the imposed respiratory rate (controlled respiration
at different respiratory rates).

1283

tern (MP)[14]. The percentage of MP (MP%) is calculated by

dividing the amount of MPs by the number of all possible patterns (i.e.,
). As the distribution of the patterns depends
on and , MP% is also a function of and .
C. Shannon Entropy
SE is defined [20] as
(2)
represents the probability of the pattern
apwhere
proximated by its sample frequency and the sum is extended to
all different patterns . SE is an index describing the shape of
the distribution of the patterns . Indeed, SE is large if the dispatterns are identically distributed
tribution is flat (all the
and the series carries the maximum amount of information). On
the contrary, SE is small if there is a subset of patterns more
likely, while others are missing or infrequent (e.g., in a Gaussian
distribution). Obviously, the SE is derived directly from the PDF
of .
D. CE and Regularity Index
The CE is defined [20] as

II. COMPLEXITY MEASURES

A. Coarse Graining and Pattern Construction
is first
Each sample of the series
normalized by subtracting the mean and, then, divided by the
standard deviation, thus obtaining the series
expressed in adimensional units. Next, the full range
) is divided into a fixed number
of dynamics (i.e.,
of values (the quantization levels) labeled with numbers from
. The quantization procedure performs a coarse
zero to
graining of the dynamics with a resolution equal to
, thus rendering the signal a sequence of symbols
from the limited alphabet of symbols
. From the symbolic series , patterns of
delayed samples (also referred to as words in terms of symbolic
.
dynamics) are constructed as
As the time delay between samples is equal to one,
sents an authentic feature of signal (i.e., a wavelet).

repre-

B. Probability Density Function of the Patterns

The wavelet

can be codified in decimal format as

(1)
again a series of inthus rendering the sequence of wavelets
with ranging
teger numbers
. With this definition, the
from zero to
probability density function (PDF) of provides the distribution of the patterns . It reports the sample frequency of each
as a function of the pattern decimal code . If the pattern
is never detected in , then it is referred to as missing pat-

(3)
is the probability of the pattern ,
where
is the probability of the most recent symbol of the pat(i.e.,
) when the previous
ones (i.e.,
tern
) are given. The inner sum can
be interpreted as the SE calculated over the distribution of the
conditioned by the patterns
and performed
symbols
over all different symbols , the outer sum is performed over
. If the signal is fully repetitive, it is
all different patterns
possible to find a value of

such that

(i.e.,

are completely predictable from the patterns

the symbols
) and
. If the signal is an identically disgiven
tributed white noise, the conditional distributions of
are flat independently of (i.e., the past samples never
reduce the uncertainty in the prediction of the next sample),
. If the white noise is not
thus producing a
identically distributed, the CE is again constant but the value
. On the contrary, if repetitive patterns
is smaller than
embedded in noise are present, the CE is not constant but
given
decreases as soon as the conditional distributions of
exhibit a sharp peak (i.e., the past samples are useful to
predict future dynamics). As an effect of its definition, CE is
like SE but a
not a measure of the incidence of patterns
follow each
measure of how much regularly the patterns
can be calculated also as
other in the series.
(4)
where
of the series

and
represent the SE of the PDF
obtained by considering patterns of length and

1284

IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 48, NO. 11, NOVEMBER 2001

respectively. Unfortunately, when CE is estimated over

can be the result of a
short data segments,
given the pattern
, thus producing a decrease
unique
of CE with regardless of the type of dynamics[19]. If this
, the
0. For
phenomenon occurs for every pattern
example, null CE values can be observed at very small in short
samples) of white noise. In this specific case,
segments (
is only an effect of an unreliable estimate of
due to the shortness of the data sequence. If the
might be found and a
length sequence is increased, several
different from one might be measured. Porta et
al. [19] propose a function referred to as corrected CE (CCE)
dealing with this bias. It is defined as
(5)
is the SE of the distribution of the quantized
where
and
is the percentage of patterns
found
series
only once in the data set. Equation (5) substitutes the erroneous
due to its unique appearcertainty associated to the pattern
ance with the maximum uncertainty detectable in the series
. Therefore, the CCE: 1) remains constant
measured by
in case of white noise; 2) decreases to zero in case of a fully
periodic signal; and 3) exhibits a minimum if repetitive patterns
,
are embedded in noise. If the CCE is normalized by
the normalized CCE (NCCE) ranges from zero to one. Therefore, based on the detection of the NCCE minimum, an index
[17].
of regularity is defined as
It ranges from one (maximum regularity) and zero (maximum
complexity).
III. METHODS
A. Experimental Protocol and Beat-To-Beat Variability Series
Extraction
Fifteen healthy young subjects (ten women and five men; age
range: 2432 years) were enrolled in this study. All the subjects
had no sign of organ or systemic disease. No one was taking
any medication and each subject was instructed to avoid beverages containing alcohol or caffeine in the day preceding recordings. Studies were performed between 09:00 a.m. and 11:00
a.m. All the subjects were carefully instructed and gave their
written consent. The protocol was approved by our Institutes
Review Board.
The protocol included four recordings lasting 10 min in
supine position with the subject placed on an electrically
driven tilt table. After 30 min allowed for stabilization, the
first recording was obtained at rest while the subject was
breathing spontaneously. The following three recordings in
supine position were obtained while the subject was controlling
his breathing rate at 10, 15, and 20 breaths/min according to
a metronome (CR10, CR15, and CR20, respectively). All the
subjects were trained to follow the metronome and adjust their
tidal volume in response to changes of the respiratory rate.
Finally, the last recording was obtained during spontaneous
breathing after the subject was passively moved to an upright

80 position by electrically rotating the tilt table. No episode

of syncope was observed.
Electrocardiogram (ECG) signal (lead II) and the respiratory
signal (via a nasal thermistor) were stored on a magnetic tape
(Racal Recorder, Southampton, U.K.). Next, signals were played
back from the tape and sampled at 300 Hz by means of a 12 bits
analog-to-digital board (National Instruments, Austin, TX). All
the analyzes were carried out on a PC by ad hoc C-code programs.
The QRS complex of the ECG signal was detected when the first
derivative of the ECG exceeded a user-defined fraction of the
maximum derivative. A parabolic interpolation was performed
on the R peak. The R peak was located at the maximum of
the parabola. The RR interval was defined as the time interval
between two consecutive R peaks. The detections were carefully
checked. Missed detections were manually inserted, while detections of peaks of noise were deleted. Only isolated ectopic beats
were occasionally observed. If an ectopic beat was detected,
the RR intervals ending or starting with the ectopic beat were
substituted with a linear interpolation between the most adjacent
RR intervals defined by normal R waves. The respiratory rate was
derived by spectral analysis from the respiratory signal to verify
the ability of the subject to follow the metronome.
B. Evaluation of Complexity Indexes
The reliability of the estimate of PDF increases while in.
creasing the length of data sequence with respect to
Unfortunately, increasing caused a lack of stationarity. Therewas maintained low by choosing small values for
fore,
the pattern length and for the number of quantization levels.
should
However, the number of detected patterns
in order
be larger than the number of possible patterns
to allow patterns to be found several times. Therefore, as series
of 300 cardiac beats were considered, SE and MP% were caland
. Differently,
culated with
as the regularity index was derived by using a minimization
procedure over , the pattern length was not a priori fixed
(CCE was necessary to avoid the unreliability of the CE estimate), while is again fixed to six.
C. Surrogate Data Approach and Deterministic Pattern
Detection
Due to the small number of quantization levels utilized to
codify the dynamics, some patterns might be more frequent than
others only by chance. In order to exclude this possibility a surrogate data approach was followed [21]. Fifteen surrogate data
series were obtained from the original one by randomly shuffling the samples according to 15 different white noise realizations, thus completely destroying the original power spectrum
but maintaining sample distribution. A complexity index (SE,
was comMP% or ) calculated over the original series
by
pared with those obtained from the surrogate series
using the formula [21]
(6)
and
where
dard deviation of

represented the mean and the stanover the 15 surrogate series. If

,

PORTA et al.: TOOLS TO TYPIFY COMPLEXITY IN SHORT HEART PERIOD VARIABILITY SERIES

1285

4, 4) or (3, 3, 1)]; 3) patterns with two like variations [2LV, the

three symbols formed an ascending or descending ramp, e.g.,
(1, 2, 4) or (5, 4, 3)]; and 4) patterns with two unlike variations [2UV, the second symbol was larger or smaller than the
remaining ones forming a peak or valley, e.g., (3, 4, 3) or (3, 0,
2)]. The choice of this strategy for pattern redundancy reduction was to group all possible patterns in four categories characterized by different frequency contents: 0V category collected
patterns characterized only by very slow frequencies (the pattern was constant); 1V class clustered patterns with some slow
and very slow frequencies as both a plateau and a ramp were
present; 2LV category grouped patterns typified only by slow
frequencies without slower frequencies as the plateau was not
codified; and 2UV class collected patterns dominated by faster
frequencies as they actually represented peaks or valleys.
E. Statistical Analysis
One way ANOVA (Bonferroni test) was utilized to test differences between rest and all the other experimental conditions.
A
was considered significant.
IV. RESULTS
A. Complexity Measures in Short Beat-To-Beat Variability of
RR Interval

Fig. 1. Examples of patterns belonging to 0V [first row: (4, 4, 4) and (1, 1, 1)],
1V [second row: (3, 4, 4) and (3, 3, 1)], 2LV [third row: (1, 2, 4) and (5, 4, 3)]
and 2UV [forth row: (3, 4, 3) and (3, 0, 2)] categories.

could not derive from a white noise with the same sample
.
distribution of the original series
Moreover, in order to allow a reliable classification of the
even on short data segpatterns lasting three beats
ments, the PDF of patterns calculated over the original segment
was compared with the PDF obtained from the surro. If, for a given pattern,
was larger
gate series
then that specific pattern was
than
not detected by chance in the series and, therefore, it was referred to as deterministic pattern.
D. Classification of Frequent Deterministic Patterns and
Redundancy Reduction
A deterministic pattern observed with a probability larger
than 0.04 in the original series, was defined as frequent deterministic pattern (FDP). Three-beat FDP patterns were reduced
into the following four families (Fig. 1): 1) patterns with no variation [0V, all the three symbols were equal forming a three-beat
plateau, e.g., (4, 4, 4) or (1, 1, 1)]; 2) patterns with one variation [1V, two symbols were equal and consecutive forming a
two-beat plateau, while the remaining one was different, e.g., (3,

An example of heart period variability series at rest, during

tilt, CR10, CR15 and CR20 are depicted in Fig. 2(a)(e), respectively. It was easy to recognize deterministic, repetitive patterns
with different durations (the slowest ones during tilt and progressively faster from CR10 to CR20). However, the duration
and the amplitude of these patterns changed thus preventing the
strict repetition of the same pattern (this phenomenon was evident even during tilt and CR10). Also the presence of irregular
oscillations with a period larger than the duration of these patterns contributed to the complexity of the series (more evident at
rest and during CR15 and CR20). Quantized series maintained
these features [Fig. 2(f)(j)]. Fig. 3 depicts the PDFs calculated
over the series of Fig. 2. At rest, the PDF [Fig. 3(a)] revealed
that some patterns were more frequently detected (the peaks
had different heights) and others were missing (the sample frequency was 0). During tilt [Fig. 3(b)], the frequency of some
patterns largely increased (some peaks emerged) as well as the
number of missing patterns. During CR10, CR15, and CR20,
PDFs [Fig. 3(c)(e)] were not different from those at rest. Fig. 4
shows the CCE functions calculated over the series of Fig. 2. At
rest [Fig. 4(a)], the RR interval was regular (the CCE exhibited
a minimum) but the CCE minimum was largely different from
zero (i.e., the regularity index was different from 1). During
tilt and CR10 [Fig. 4(b) and (c), regularity increased (the CCE
minimum decreased], while it was unchanged during CR15 and
CR20 [Fig. 4(d) and (e)]. Fig. 5 shows an RR interval series at
rest [Fig. 5(a)] and its surrogate series [Fig. 5(b)]. Fig. 5(c) depicts the PDFs calculated over the original and surrogate series
(open and filled bars, respectively). PDF obtained from original
series was different from that obtained from its surrogate series. Indeed, several peaks were larger and missing patterns were
more present. However, PDF calculated over the surrogate data
was not flat (some patterns could be frequently detected only

1286

IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 48, NO. 11, NOVEMBER 2001

Fig. 2. Beat-to-beat series of heart period variability in a healthy young subject at rest (a), during tilt (b), CR10 (c), CR15 (d) and CR20 (e). Their relevant
quantized series are depicted below (f,g,h,i,j). Slow oscillations (at 0.1 Hz) are clearly evident in (b), while high-frequency rhythms are present in (a), (c), (d), (e),
progressively faster from (c) to (e). Quantized series maintain these features.

Fig. 3. PDFs calculated on the series of Fig. 2. In all the experimental conditions the PDF is not flat, thus indicating that some patterns are more frequently
detected than others. SE is 4.02, 3.30, 3.99, 4.36, and 4.57, while MP% is 62.96, 79.63, 64.35, 52.78, and 45.37, at rest, during tilt, CR10, CR15, and CR20,
respectively. The SE is significantly reduced during tilt (b).

Fig. 4. CCEs calculated on the series of Fig. 2. In all the experimental conditions the CCEs exhibit a minimum, thus detecting a certain level of regularity. The
CCE minimum is deeper during tilt (b) and CR10 (c). The regularity index  is 0.175, 0.482, 0.292, 0.154, and 0.117 at rest, during tilt, CR10, CR15, and CR20,
respectively.

by chance). Fig. 5(d) depicts CCEs calculated over the original

and surrogate series of Fig. 5(a) and (b). The CCEs calculated
over the original (dotted line) and the surrogate data (solid line)
were different: the former exhibited a deep minimum, while the
latter was completely flat.
The results of complexity analysis are summarized in Table I.
The complexity indexes calculated over all the original series
from those obtained from their surwere different
rogates in all the experimental conditions. During tilt, SE de, while MP% increased
. Under
creased

controlled respiration SE and MP% did not change. Regularity

index increased during tilt
and CR10
,
while it was not influenced by CR15 and CR20.
B. Classification of FDPs in Short Beat-To-Beat Variability
of RR Interval
The procedure to detect FDPs is described in Fig. 6. It shows
the superposition of a PDF of an original RR interval series at
rest (open bars) and the average PDF plus two times the standard deviation calculated over 15 surrogate realizations (filled

PORTA et al.: TOOLS TO TYPIFY COMPLEXITY IN SHORT HEART PERIOD VARIABILITY SERIES

Fig. 6.

1287

Description of the procedure to detect FDPs. A FDP is found if the

PDF calculated over the original series (open bars) overcomes av[PDF ] +
2 sd[PDF ] calculated over 15 realizations of surrogate data (filled bars) and
if PDF is larger than 0.04 (dotted line). Only three FDPs are detected.
1

PERCENTAGE

OF

TABLE II
EXPERIMENTAL SUBJECTS EXHIBITING FREQUENT
DETERMINISTIC PATTERNS (FDP)

Fig. 5. Original beat-to-beat variability of heart period at rest (a) and its
surrogate series (b). The PDFs calculated over the original beat-to-beat series
of heart period at rest (open bars) and over its surrogate series (filled bars)
are depicted in (c). The PDF obtained from the surrogate data is not flat, thus
indicating that several patterns can be repetitive only by chance. SE is very
different (3.947 and 4.607 over original and surrogate series, respectively) as
well as MP% (64.81 and 42.13, respectively). The CCEs calculated over the
original (dotted line) and surrogate (solid line) series are depicted in (d). The
CCE calculated over the surrogate series is completely flat (the regularity index
is 0.269 and 0.006 over the original and surrogate series, respectively).
TABLE I
RESULTS OF THE COMPLEXITY ANALYSIS

bars). The open bar was higher than filled bar when the pattern
was more frequently detected in the original than in surrogate
data. That pattern was referred to as deterministic pattern. If the
sample frequency of the deterministic pattern was larger than
0.04 (the dotted line), it was referred to as FDP. Three FDPs are
detectable in Fig. 6.
Table II reports FDPs and their class of belonging according
to the adopted redundancy reduction criterion (first and second
column, respectively) and percentage of experimental subjects

that exhibit that particular pattern while varying the experimental condition (the remaining columns). Only FDPs found
in more than two subjects and at least in one experimental
condition were reported. Therefore, if a FDP was missing in the
first column of Table II, this means that it was strongly unlikely
in all the experimental conditions. A hyphen was reported in
Table II when the relevant FDP was found in less than three
subjects in that specific experimental condition. At rest FDPs
mainly belonged to three classes (i.e., 0V, 1V, and 2LV) and the
1V class was privileged. During tilt two classes were detected
(i.e., 0V and 1V) and the 0V class was more important. During
CR10 and CR15, two classes were identified: 1V and 2LV
classes during CR10 and 0V and 1V classes during CR15.
During CR20 the 2UV class was found with the classes 0V
and 1V. Table III reports the percent change of the number
of subjects that exhibit FDPs belonging to the class indicated

1288

IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 48, NO. 11, NOVEMBER 2001

TABLE III
PERCENT CHANGES OF THE NUMBER OF SUBJECTS EXHIBITING FREQUENT
DETERMINISTIC PATTERNS BELONGING TO THE CONSIDERED CLASS

could be covered in an irregular fashion, thus explaining a high

approximated entropy in presence of a small SE. Also Fig. 5 illustrates this concept. Indeed, the distribution pattern obtained
from a random surrogate data (Fig. 5(c), filled bars) is not flat (a
large amount of missing patterns are present, thus limiting the
number of patterns actually present and the SE) but the regularity index is null (the CCE is flat, Fig. 5(d), solid line).
B. Complexity Analysis of Heart Period Variability

During tilt, we observed a dramatic increase of number of

subjects exhibiting patterns relevant to the 0V class, while the
2LV and 2UV classes diminished their importance. During
CR10, 1V, and 2LV patterns were detected more frequently,
while the number of subjects characterized by 2UV patterns
largely decreased. During CR15, the increase of the importance
of 1V and 2LV classes was less remarkable than during CR10.
Also the reduction of the number of subjects exhibiting 2UV
patterns was less important. During CR20, the importance of
the 0V, 1V, and 2LV categories decreased while the number of
subjects characterized by 2UV patterns dramatically increased.
V. DISCUSSION
The proposed approach to the complexity analysis of heart
period variability recommends: 1) to jointly consider entropy
and entropy rate indexes to quantify two different aspects of
complexity; 2) to perform pattern classification to clarify the
type of patterns contributing to the generation of complexity.
A. Entropy and Entropy Rate Indexes Provide Different
Information
SE is a measure of the complexity of the pattern distribution.
The presence of peaks in the pattern distribution (relevant to patterns more frequently detected) or valleys (relevant to missing
or less frequent patterns) determines the decrease of the SE with
respect to its maximum value provided by a flat distribution.
Therefore, the SE correlates with the percentage of missing patterns. The CE [17] and the approximated entropy [16] provide
measures of the complexity of the dynamical relationship between a pattern and the next one. If the temporal sequence of
the patterns is fully regular (i.e., the patterns follow each other
in a repetitive periodic way), the CE is zero. On the contrary,
the maximum value of the CE is found when no relationship
between a pattern and the next one is found (the sequence of
pattern is completely random). Due to these different definitions, measures of complexity based on entropy (e.g., SE) and
entropy rates (e.g., approximated and CE) provide different information. Indeed, a significant increase in the regularity index
(a decrease of CE) does not imply a decrease in the SE, as it can
be observed during CR10. It appears that heart period variability
during CR10 is characterized by the same amount of patterns
found at rest (during CR10 MP% is not significantly different
from rest) but the temporal sequence of patterns is more regular
and predictable (i.e., the next pattern is selected in a less random
fashion). Conversely, a significant decrease of the SE does not
imply a decrease in the entropy rate (e.g., the approximated entropy) as it was observed by Palazzolo et al. [3] in dogs during
standing. It appears that the limited number of frequent patterns

At rest, short variability series of heart period exhibit a large

and a level of regunumber of missing patterns
, thus evidencing that
larity largely different from
heart period variability contains a limited amount of patterns
but the temporal sequence of these patterns is not fully regular
(Table I). During tilt, the decrease of the SE and the increase
of missing patterns and of the regularity index suggest that the
decrease in complexity is the effect of a presence of a smaller
number of patterns, the temporal sequence of which is more predictable. During CR10, the increase in regularity not associated
with a decrease of the SE and of the number of missing patterns demonstrates that the same amount of patterns found at
rest forms a more repetitive sequence of patterns. During CR15
and CR20 the SE, the number of missing patterns and the regularity index do not change, thus measuring the same level of
complexity calculated at rest both in terms of complexity of the
pattern distribution and of the relationship linking a pattern to
the next one.
In a previous paper [17], we attributed the increase of
regularity found during tilt to the synchronizing action of the
sympathetic activation on mechanisms modulating heart period
in the low-frequency (LF) band (from 0.04 to 0.14 Hz) and to the
limiting action over the high-frequency oscillation (HF, at the
respiratory rate), thus reducing the number of temporal scales
present in heart period variability. As a new finding, this study
points out that the sympathetic activation is also able to reduce
complexity through a reduction of the number of different
patterns constituting heart period variability. In a previous
paper [17], we attributed the frequency-dependent effect of
controlled respiration on regularity of heart period (only slow
breathing rates produce an increase of regularity) to the tidal
volume (it decreases while increasing the respiratory rate).
Indeed, slow breathing rates are more effective in stimulating
cardiac and pulmonary low pressure receptors and in inducing
respiratory-related changes on arterial pressure able to produce
heart rate variations via the baroreflex feedback. As a new
finding, this study indicates that controlled respiration leaves
unchanged the number of patterns composing heart period
variability and, therefore, the flexibility of the cardiovascular
regulatory system even in presence of a more regular alternation
among patterns produced by slower breathing rates. Pathological
conditions (e.g., at the level of low pressure receptors or neural
pathways) could reduce the ability of slower breathing rates
to regularize heart period.
C. Pattern Classification in Heart Period Variability Series
The SE provides a measure of the complexity of the pattern
distribution but does not furnish any indication about the type

PORTA et al.: TOOLS TO TYPIFY COMPLEXITY IN SHORT HEART PERIOD VARIABILITY SERIES

of patterns detected in the series. Indeed, if the dynamics of the

two series are characterized by different patterns with identical
sample frequencies, the SE is equal. Therefore, different dynamics may exhibit equal SE, thus rendering necessary a pattern
classification to understand which patterns are actually involved
in generating the complexity.
Usually pattern analysis in heart period variability series is
based on an a-priori definition of the patterns to be detected
[1], [22]. This approach needs to define the physiological correlate of the searched wavelet (e.g., the tachicardia at the onset
of physical activity [22]) to ensure the search for deterministic
and meaningful patterns. In this study, we propose a new approach based on the identification of the most frequent features
directly on heart period variability series. This approach exploits
the pattern classification performed to calculate the sample frequency of each pattern necessary to derive both the Shannon
and conditional entropies. This approach requires: 1) to validate the deterministic nature of the pattern; 2) to discard unlikely patterns; 3) to provide a redundancy reduction criterion to
group the detected patterns into a limited number of categories;
and 4) to a-posteriori search for physiological correlates of categories. To validate the deterministic nature of a pattern, we use
a surrogate data approach. When a coarse graining procedure
is applied to convert the original series in a sequence of symbols belonging from a very limited alphabet (six symbols in this
application), it is mandatory to perform surrogate data analysis
in order to check whether some patterns are frequent only by
chance. Unlikely deterministic patterns are discarded by setting
a threshold on the sample frequency (0.04 in this study), thus
classifying only FDPs (Table II) and focusing only on wavelets
capable to describe a large amount of the dynamics of the series.
This choice provides the first redundancy reduction criterion utilized in this study. A second redundancy criterion is applied to
group FDPs in four categories based on the number and type
of symbol variations. We find out that heart period variability
series have different compositions in terms of the four defined
categories (Table II) and changes of the importance of the four
classes allow to distinguish the considered experimental conditions (Tables II and III).
The different weight of the four classes can be interpreted
by considering the different frequency content of the heart period variability under the considered experimental conditions
[23], [24]. Indeed, 0V patterns are features of slow waves (e.g.,
LF oscillations) while 2LV and 2UV patterns are fragments of
faster waves (e.g., HF oscillations). At rest mainly three classes
(0V, 1V and 2LV) are detected due to the contemporaneous
presence of LF and HF oscillations. During tilt, the large increase of number of subjects showing 0V patterns and the important decrease of the number of subjects exhibiting 2LV and
2UV patterns can be explained in terms of the increased presence of LF waves and a decreased presence of HF oscillations.
During CR10, the number of subjects showing 0V patterns decreases, while 2LV patterns are more likely as a result of the
absence of LF oscillations and of the presence of HF oscillations. In addition, during CR10, 2UV patterns are largely less
likely as respiratory sinus arrhythmia is stable and not too fast
Hz). During CR15 the composition of heart rate vari(

1289

ability in terms of the defined four categories is similar to that

observed at rest. During CR20, features characterizing fast frequencies (2UV patterns) are more likely, while 0V patterns relevant to slow oscillations are also present, thus demonstrating
the rise of oscillations slower than HF rhythms.
Even though sensible to the frequency content of the series
(i.e., to the linear features of the signal), the proposed classification method cannot be considered as another approach to perform power spectral analysis. Indeed, the approach takes into
account also nonlinear features and completely decomposes the
series in patterns not limited to be portions of sinusoids. At
the moment, we lack the necessary physiological knowledge to
transform the detected patterns or categories into physiological
correlates and to combine the detected patterns or categories
forming longer, nonlinear and complex waveforms. However,
several findings not detectable trough conventional methods and
typically related to nonlinear features, can be derived from the
classification of FDPs (Table II): 1) during tilt, all the patterns
belonging to the 0V class are frequently detected (only (0, 0,
0) is unlikely), thus pointing out the importance of amplitude
modulations in this experimental condition; 2) during CR10,
increasing ramps [i.e., (2, 3, 4)] are not found as frequently
as decreasing ramps [i.e., (4, 3, 2)], thus suggesting an asymmetry in the respiratory sinus arrhythmia; 3) patterns describing
abrupt changes in the heart period ([i.e., variations larger than
one quantization level like in (1, 3, 3), (5, 4, 1), or (5, 3, 5)] are
not repetitively found, although they are present, thus largely
contributing to the generation of heart period complexity; and 4)
during CR15 and CR20, patterns, that are composed by quantization levels below three [e.g., (3, 2, 2)], are unlikely, thus suggesting a more regular dynamics at larger heart periods.
VI. CONCLUSION
As indexes based on entropy (e.g., SE) and entropy rate (e.g.,
CE) provide different information, they should be utilized in
association to evaluate complexity of heart period variability.
Moreover, as complexity measures based on entropy calculation
do not provide any indication about the pattern types involved
in the generation of a specific level of complexity, measures
of complexity should be accompanied by a pattern classification procedure. Pattern classification, when based on surrogate
data approach and redundancy reduction criteria, help to identify physiological conditions characterized by the activation of
different mechanisms responsible for cardiovascular regulation.
In presence of physiological correlates of the detected patterns,
the proposed approach can be extremely selective in searching
for a specific status and in performing a waveform analysis. For
example, in the study of arrhythmias, where specific and well
codified sequences of cardiac beats are clearly related to malignant events, this tool may be extremely useful.
REFERENCES
[1] H. Bettermann, D. Amponsah, D. Cysarz, and P. Van Leeuwen, Musical rhythms in heart period dynamics: A cross-cultural and interdisciplinary approach to cardiac rhythms, Amer. J. Physiol., vol. 277, pp.
H1762H1770, 1999.

1290

IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 48, NO. 11, NOVEMBER 2001

[2] N. Iyengar, C.-K. Peng, R. Morin, A. L. Goldberger, and L. A. Lipsitz,

Age-related alterations in the fractal scaling of cardiac interbeat interval
dynamics, Amer. J. Physiol., vol. 271, pp. R1078R1084, 1996.
[3] J. A. Palazzolo, F. G. Estafanous, and P. A. Murray, Entropy measures
of heart rate variation in conscious dogs, Amer. J. Physiol., vol. 274,
pp. H1099H1105, 1998.
[4] S. M. Pikkujamsa, T. H. Makikallio, L. B. Sourander, I. J. Raiha, P.
Puukka, J. Skytta, C.-K. Peng, A. L. Goldberger, and H. V. Huikuri,
Cardiac interbeat interval dynamics from childhood to senescence.
Comparison of conventional and new measures based on fractals and
Chaos theory, Circulation, vol. 100, pp. 393399, 1999.
[5] J. S. Richman and J. R. Moorman, Physiological time-series analysis
using approximate entropy and sample entropy, Amer. J. Physiol., vol.
278, pp. H2039H2049, 2000.
[6] S. Vikman, T. H. Makikallio, S. Yli-Mayry, S. M. Pikkujamsa, A.-M.
Koivisto, P. Reinikainen, K. E. J. Airaksinen, and H. V. Huikuri, Altered
complexity and correlation properties of RR interval dynamics before
the spontaneous onset of paroxysmal atrial fibrillation, Circulation, vol.
1000, pp. 20792084, 1999.
[7] J. L. Feldman, J. C. Smith, H. H. Ellenberger, C. A. Connelly, G. Liu,
J. J. Greer, A. D. Lindsay, and M. R. Otto, Neurogenesis of respiratory
rhythm and pattern: Emerging concepts, Amer. J. Physiol., vol. 259, pp.
R879R886, 1990.
[8] G. Preiss and C. Polosa, Patterns of sympathetic neuron activity associated with Mayer waves, Amer. J. Physiol., vol. 226, pp. 724730, 1974.
[9] B. Anderson, R. A. Kenney, and E. Neil, The role of the chemoreceptors
of the carotid and aortic regions in the production of Mayer waves, Acta
Physiol Scand, vol. 20, pp. 203220, 1950.
[10] A. C. Guyton and J. H. Harris, Pressoreceptor-autonomic oscillation:
A probable cause of vasomotor waves, Amer. J. Physiol., vol. 165, pp.
158166, 1951.
[11] A. Malliani, Principles of Cardiovascular Neural Regulation in Health
and Disease. Norwell, MA: Kluwer Academic, 2000.
[12] J. U. Meyer, L. Limdbom, and M. Intiglietta, Coordinated diameter oscillations at arteriolar bifurcation in skeletal muscle, Amer. J. Physiol.,
vol. 253, pp. H568H573, 1987.
[13] A. L. Goldberger, Non-linear dynamics for clinicians: Chaos theory,
fractals and complexity at the bedside, Lancet, vol. 347, pp. 13121314,
1996.
[14] A. Voss, J. Kurths, H. J. Kleiner, A. Witt, N. Wessel, P. Saparin, K. J.
Osterziel, R. Schurath, and R. Dietz, The application of methods of
nonlinear dynamics for the improved and predictive recognition of patients threatened by sudden cardiac death, Cardiovasc. Res., vol. 31,
pp. 419433, 1996.
[15] D. Cysarz, H. Bettermann, and P. Van Leeuwen, Entropies of short binary sequences in heart period dynamics, Amer. J. Physiol., vol. 278,
pp. H2163H2172, 2000.
[16] S. M. Pincus, Heart rate control in normal and aborted-SIDS infants,
Amer. J. Physiol., vol. 33, pp. R638R646, 1993.
[17] A. Porta, S. Guzzetti, N. Montano, M. Pagani, V. K. Somers, A. Malliani,
G. Baselli, and S. Cerutti, Information domain analysis of cardiovascular variability signals: Evaluation of regularity, synchronization and
co-ordination, Med Biol Eng Comput, vol. 38, pp. 180188, 2000.
[18] S. M. Pincus and A. L. Goldberger, Physiological time-series analysis: What does regularity quantify, Amer. J. Physiol., vol. 266, pp.
H1643H1656, 1994.
[19] A. Porta, G. Baselli, D. Liberati, N. Montano, C. Cogliati, T. GnecchiRuscone, A. Malliani, and S. Cerutti, Measuring regularity by means of
a corrected conditional entropy in sympathetic outflow, Biol. Cybern.,
vol. 78, pp. 7178, 1998.
[20] A. Papoulis, Probability, Random Variables and Stochastic Processes. New York: McGraw-Hill, 1984.
[21] J. Theiler, S. Eubank, A. Longtin, and J. Galdrikian, Testing for nonlinearity in time series: The method of surrogate data, Physica D, vol.
58, pp. 7794, 1992.
[22] D. Roach, P. Malik, M. L. Koshman, and R. Sheldom, Origins of heart
rate variability: Inducibility and prevalence of a discrete, tachicardic
event, Circulation, vol. 99, pp. 32793285, 1999.
[23] T. E. Brown, L. A. Beightol, J. Kob, and D. L. Eckberg, Important
influence of respiration on human RR interval power spectra is largely
ignored, J. Appl. Physiol., vol. 75, pp. 23102317, 1993.
[24] M. Pagani, F. Lombardi, S. Guzzetti, O. Rimoldi, R. Furlan, P. Pizzinelli,
G. Sandrone, G. Malfatto, S. DellOrto, E. Piccaluga, M. Turiel, G.
Baselli, S. Cerutti, and A. Malliani, Power spectral analysis of heart
rate and arterial pressure variabilities as a marker of sympatho-vagal interaction in man and conscious dog, Circ. Res., vol. 59, pp. 178193,
1986.

Alberto Porta was born in 1964. He graduated in

electronic engineering from Politecnico di Milano,
Milan, Italy, in 1989. He received the Ph.D. degree
in Biomedical Engineering at Politecnico di Milano,
Milano, in 1998.
He was a Research Fellow on Automatic Control
and System Theory at the Dipartimento di Elettronica
per lAutomazione, Universita di Brescia, Brescia,
Italy, from 1989 until 1994. Since 1999, he has
been with the Dipartimento di Scienze Precliniche,
Universita degli Studi di Milano, Milan, Italy. Since
1999, he has taught a course on biomedical instrumentation at the Dipartimento
di Elettronica per lAutomazione, University of Brescia, Brescia, Italy. His
primary interests include time series analysis, nonlinear dynamics, system
identification, and modeling applied to cardiovascular control mechanisms.

Stefano Guzzetti was born in 1954. He graduated in

medicine and surgery from the University of Milan,
Milan, Italy, in 1979 (magna with laude). He was specialized in cardiology in 1981 and in medical statistics in 1984 at the University of Milan.
He is presently Senior Registrar with the Division
of Internal Medicine II at the L. Sacco Hospital,
Milan, Italy. He is the Referent of the Heart Failure
outpatient clinic at the L.Sacco Hospital. He published more than 60 scientific papers in the areas of
the neural regulation of cardiovascular functions with
particular attention to the neural physiopathological mechanisms involved in
heart failure.

Nicola Montano was born in 1963. He graduated in

medicine at the University of Milan, Milan, Italy, in
1988. He received the Ph.D. degree in clinical physiopathology from the University of Milan in 1994.
He is presently Established Investigator of Internal
Medicine at the University of Milan. His research interests are in the field of neural control of cardiovascular function and are mainly related to recordings
of neural autonomic activities in animal and humans
and to the relationship between neural and cardiovascular oscillatory patterns.

Raffaello Furlan was born in 1954. He graduated in

medicine from the University of Milan, Milan, Italy.
Presently, he is Senior Registrar in Internal
Medicine and Nontenure Professor of the postgraduate course in clinical psychology at University of
Milan, since 1991. From 1977 to 1984, he took part
in animal studies aimed at evaluating the role of
cardiac and vascular sympathetic excitatory reflexes
regulating systemic arterial pressure. Since 1982, he
is involved in clinical studies aimed at assessing the
changes in the neural mechanisms controlling the
cardiovascular system in different physiological and pathophysiological conditions including the gravitational stimulus, physical exercise, athletic training,
shift-work, syncope, dysautonomia, active ulcerative colitis, and others. During
a sabbatical at the Clinical Research Center of the Vanderbilt University,
Nashville, TN, in 1995, he was External Advisor for a program project grant
aimed at clarifying the pathophysiology of chronic orthostatic intolerance,
which was funded the following year. He is in charge of the Syncope and
Orthostatic Intolerance Unit which is part of an international web, organized
by the National Dysautonomia Research Foundation (NDRF), linking centers
with recognized expertise in the different forms of dysautonomia.

PORTA et al.: TOOLS TO TYPIFY COMPLEXITY IN SHORT HEART PERIOD VARIABILITY SERIES

Massimo Pagani is Professor of Internal Medicine

at the University of Milano Medical School, Milan,
Italy. His main interests are neural control of the circulation, examined in intact conscious animals and
men, and computer applications to assess invasively
and noninvasively hemodynamic performance of the
heart and of the vasculature. He has contributed to
clinical applications of computer analysis of cardiovascular variabilities in vaious fields as health promotion in ambulant patients, prognostic evaluation in
coronary artery disease, and telematics application to
arterial hypertension treatment.

Alberto Malliani was born in 1935. He graduated in

medicine from the University of Siena, Siena, Italy in
1959.
From 1965 to 1967, he was Fellow of the Public
Health Service at Columbia University, New York.
During the early 1970s, he was Visiting Professor at
universities in Utah and Texas.
He is presently Full Professor of Internal Medicine
at the University of Milan and is the head of the Department of Internal Medicine II at the L. Sacco
Hospital. He has published more than 200 full papers
that have appeared in the most prestigious journals and in textbooks such as
Handbook of Physiology and Textbook of Pain. Among these articles, editorials
in Circulation, the American Heart Journal, the British Heart Journal, and Cardiovascular Research and other minor Journals. His articles have received more
than 6500 quotations in current literature.
Dr. Malliani received the award for Student of the Year in the Faculty of
Medicine from the University of Siena in 1959. In 1980, the European Society
for Clinical Investigation awarded Dr. Malliani the Mack-Foster Award. He
is President of the Societ Italiana di Medicina Interna. He was President of
the XXI International Congress of Neurovegetative Research. He is a member
of the Editorial Board of Circulation, of the Autonomic Neruroscience and of
other minor journals.

1291

Sergio Cerutti (M81SM97) is Professor and

Chairman of Biomedical Engineering (BME) of
the Department of Biomedical Engineering at the
Polytechnic University, Milan, Italy. He is Chairman
of the Bachelor Degree (Diploma Universitario) in
BME, and Chairman of the Master Degree (Laurea)
in BME at the same University. He has also taught
courses at a graduate level on biomedical signal
processing at engineering faculties (Milano and
Roma) and at specialization schools of medical
faculties (Milano and Roma).
Dr. Cerutti was an Elected Member of IEEE-EMBS AdCom (Region 8) from
19931996. He has been member of the International Committee of the various Annual Conferences of EMB Society since 1988. He is a member of the
Steering Committee of the IEEE-EMBS Summer School on Biomedical Signal
Processing; he was the local organizer of the I and III IEEE-EMBS Summer
Schools held in Siena, Italy, on July 1995 and on June 1999, respectively. He is
a Member of Regional Conference Committee of IEEE-EMBS. He is an Associate Editor of IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING and serves
on the EditorialBoard of various other scientific international journals. In the
last ten years he has published more than 200 papers in the area of biomedical
engineering [main topics: biomedical signal processing (ECG, blood pressure
signal and respiration, cardiovascular variability signals, and EEG and evoked
potentials) and cardiovascular modeling, neurosciences, medical informatics,
etc].