Smart Health 21 (2021) 100179

Contents lists available at ScienceDirect

Smart Health
journal homepage: www.elsevier.com/locate/smhl

A smart point-of-care compliance monitoring solution for brace

treatment of adolescent idiopathic scoliosis patients☆
Omid Dehzangi a, *, Bhavani Anantapur Bache b, Omar Iftikhar c, Jeffrey Wensman d,
Ying Li e
a
Rockefeller Neuroscience Institute, West Virginia University, 33 Medical Center Drive, 8 Medical Center Drive, Morgantown, 26505, USA
b
Department of Electrical and Computer Engineering, University of Michigan, Dearborn, MI, 48128, USA
c
Department of Computer and Information Science, University of Michigan, Dearborn, MI, 48128, USA
d
Orthotics and Prosthetics Center, University of Michigan, 2850 S.Industrial Hwy, Ann Arbor, MI, 48104, USA
e
C.S. Mott Children’s Hospital, University of Michigan, Ann Arbor, MI, 48109, USA

A R T I C L E I N F O A B S T R A C T

2000 MSC: Scoliosis is a medical condition which occurs in adolescents, where an individual’s spine develops
41A05 curvature. A Thoracolumbosacral orthosis (TLSO) is a type of brace used as a long term treat-ment
41A10 method to control the lateral curvature of the spine in adolescent idiopathic scoliosis (AIS).
65D05
However, compliance of the daily prescribed duration and tightness of the brace is a big chal­
65D17
lenge, which plays a significant role in the success of brace treatment. In this paper, we designed
and de-veloped a wearable multi-modal sensor solution, which is embedded into the AIS patient’s
brace. The custom-designed hardware consists of a sensor board, a force sensor & housing, a
miniature triaxial accelerometer and gyroscope sensor. The force sensor records the force being
exerted by the patient’s brace padding, while the accelerometer and gyroscope sensors capture
upper trunk movements, providing cues to determine the patient’s activities and lifestyle to help
with more accurate estimation of the two key parameters, including 1) the in-brace tightness level
2) the duration of the brace wear, in an unsupervised manner. Furthermore, we presented a signal
pro-cessing and data analytic methodology to identify the duration and tightness of the brace
wear, in addition to identifying various patient activities based on the fusion of continuous force
and iner-tial motion sensor recordings. We, specifically, derived a signal processing activity
identification method using power spectral density estimation of the sensor data recordings.
Then, we evaluate of the effectiveness of brace treatment in a pervasive manner. The proposed
method evaluates the duration of brace wear through the process of segmentation and calculates
the tightness level of the brace by estimating the baseline force per segment of data. This analysis
is done when the patient performs activities including sitting, standing, climbing, walking,
running and lying. We investigated an experimental scenario in which the patient performs a
series of pre-defined activ-ities at home during day-long segments of brace wear. All analysis is
performed using pervasive sensor data recordings. The experimental results demonstrated that we
achieved an overall accu-racy of 99.8%, 100% and 99.9% for patients 1, 2 and 3 respectively for
semi-supervised activity detection. We continued our compliance study for approximately a
month. The level of tight-ness of brace-fit gradually reduced over a period of 2 weeks by 30% as

☆
y This is an example for title footnote coding.
* Corresponding author.
E-mail addresses: oiftikha@aumich.edu (O. Dehzangi), bbache@aumich.edu (B.A. Bache), jwensman@med.umich.edu (J. Wensman), yingyuli@
med.umich.edu (Y. Li).

https://doi.org/10.1016/j.smhl.2021.100179

Available online 26 March 2021

2352-6483/© 2021 Published by Elsevier Inc.
O. Dehzangi et al. Smart Health 21 (2021) 100179

the compliance of brace treatment increased from 7% to 90% on average. Given our experimental
results and objective observations, our proposed system is capable of arranging for re-fitting
sessions automatically so that the physician can adjust the brace tightness levels for a more
effective brace treatment.

1. Introduction

Idiopathic scoliosis is an abnormal curvature of the spine that can worsen throughout growth, making it vital for doctors to treat it
during the early stages. For that reason, physicians focus on skeletally immature adolescents who show signs and symptoms of
idiopathic scoliosis. Today, 3% of children below the age of 16 years are diagnosed with adolescent idiopathic scoliosis (AIS). The
severity of scoliosis in a patient is measured by the Cobb angle. The Cobb angle refers to magnitude of the spinal curvature as measured
on a posteroanterior plain radiograph of the spine. The most common form of treatment of AIS in skeletally immature individuals is a
thoracolumbosacral orthosis (TLSO). While the brace does not provide a solution to fix the curvature, it helps prevent worsening of the
curve throughout the adolescent’s growth. If progression of the spinal curvature is prevented, then surgery can be avoided and this is
considered a brace treatment success Herring et al. (2010). In order to ensure that the treatment is successful, it is important to monitor
the quality and duration of brace wear. The total number of hours of brace wear correlates to the lack of curve Herring et al. (2010)
Rahman et al. (2005). A braced curve that remains ≤45◦-50◦ at skeletal maturity is considered a treatment success, as bracing is no
longer effective once patients are skeletally mature. Curves that are ≤45◦-50◦ at skeletal maturity are not likely to progress during
adulthood Asher and Burton (2006). Several studies provided convincing evidence about the effectiveness of this treatment given
appropriate usage Sapountzi-Krepia et al. (2006). A Dose-response curve has been demonstrated by Katz et al. and Weinstein et al., in
which duration of brace wear is positively associated with the rate of treatment success Herring et al. (2010) Dolan et al. (2014).
Weinstein et al. Dolan et al. (2014) reported that patients who wore the brace for 0–6 h daily had a success rate of 42%, whereas
patients who wore the brace for at least 12.9 h had success rates of 90–93% Dolan et al. (2014). The effectiveness of treatment depends
on the duration of wear and tightness of the brace. However, monitoring brace compliance is a challenging task. Previous studies have
demonstrated that when brace wear is accurately monitored with a temperature sensor, adherence to brace wear is frequently
overestimated Herring et al. (2010)–Asher and Burton (2006). Morton et al. Morton et al. (2008) found that patients actually wore the
brace for only 47% of the prescribed time, even though physicians, orthotists, parents, and patients estimated that the brace was worn
for 64%, 66%, 72%, and 75% of the prescribed time, respectively. Katz et al. reported that patients wore the brace for the same number
of hours regardless of whether the prescribed time was 16 h or 23 h Morton et al. (2008).

2. Related work

2.1. Related work on activity detection

With the increase in usage of wearable technology in the health field, the data analysis associated with these devices has helped
doctors in monitoring patient treatment. Understanding daily activities of patients affected by scoliosis is quite important to physi­
cians. In our research, we plan to monitor and evaluate the duration and quality of brace wear during daily activities of the patient by
correlating the force, acceleration and angular velocity collected by a multi-modal sensor solution. There has been extensive research
in the areas of activity detection from accelerometer and gyroscope readings. Ermes et al. developed a novel method to classify the
unsupervised data obtained from a 3D accelerometer placed on the subject’s wrist and hip. In addition, data from a GPS receiver was
recorded and analyzed for activity identification Ermes et al. (2008). The total classification accuracy achieved was 89%. Pereira et al.
developed a MHARS or Mobile Human activity recognition system to monitor di\fferent activities performed by patients. The authors
used accelerometer data, heart rate, altitude and body temperature to monitor the activities and the patient’s health status Ribeiro
Filho et al. (2016). They achieved an accuracy of 86.7% in classifying patient data into different activities. Capela et al. designed a
unique methodology which takes into account the transition between different phases of sitting, standing and lying to detect the three
activities Capela et al. (2015). They achieved an accuracy of 96% for sitting, 98.7% for standing and 100% for lying. Kwapisz et al. used
accelerometers in a mobile phone for activity detection. They used different features such as, mean acceleration for each axis, standard
deviation and absolute average difference to classify different activitiesPereira et al. (2016). The authors achieved an overall accuracy
of 91.7%. Bao et al. used a combination of 5 accelerometers including four on the limbs and one on the hip to detect different activities
Bao and Intille (2004). They achieved an overall accuracy of 84.6% for activity recognition. We implement a novel method for
pervasive annotation and segmentation of different activities and use that information to design an activity-specific based model to
estimate the quality of brace fit with the help of the force readings obtained from the inside of the brace. Additionally, we developed a
new method to extract the number of footsteps walked by the patients. Furthermore, we employed three discriminative features to
train our predictive model to achieve an overall activity identification accuracy of 100%.

2.2. Related work on monitoring treatment compliance

To properly evaluate the effectiveness of brace treatment, doctors monitor the duration of brace wear, to confirm whether the
patient is complying to their prescribed treatment. Depending on the severity of the condition, physicians recommend that the brace

2
O. Dehzangi et al. Smart Health 21 (2021) 100179

needs to be worn for up to 23 h a day for the treatment to be effective. A study by Weinstein et al. showed that TLSO treatment
significantly reduced the curve progression in patients Weinstein et al. (2013) with AIS. They found that longer hours of brace wear
proved to provide a greater benefit to the patients. Rahman et al. developed a methodology to determine the duration of brace wear by
the patient by using a temperature sensor Rahman et al. (2005). The brace was considered to be worn if the temperature recorded was
above 26 ◦ C. The duration of the brace wear was divided by 23 to calculate the percentage of compliance. Temperature sensors are
highly sensitive to minor fluctuations and temperature changes. These changes can be environmental factors such as changes in overall
temperature of the weather, or the temperature in an indoors setting. Switching on the air-conditioner in the house can result in a
temperature change. Such uncertainties can result in false values. As a result, a temperature sensor can often times over-estimate the
duration of brace wear. Moreover, it does not provide a reliable measure of brace fit quality. Karol et al. demonstrated that patients
with AIS who received regular feedback about their compliance with brace wear wore their brace for significantly more hours per day
as compared to patients who did not receive feedback. In that study, brace wear was monitored using a temperature sensor embedded
inside the brace. Patients who received their brace wear data at their follow-up appointments with the physician had improved
compliance Karol et al. (2016). In our work, we design and implement a reliable heuristic method of evaluating the compliance in
treatment. We use a more robust method by segmenting the data to provide an accurate calculation of the compliance of brace
treatment.

3. Overview of our work and contribution

In our work, we intend to discover an innovative and effective treatment-monitoring methodology by implementing a context-
aware remote sensing solution. The subject size for this study was three patients, all of whom were females in the age group of
10–13 years old. We conducted our experiments in three different scenarios: 1) supervised, 2) semi-supervised and 3) unsupervised. In
the supervised scenario, all three patients volunteered to perform pre-defined activities including sitting, standing, walking, running,
lying and climbing consecutively for 2 min per activity on a daily basis. Furthermore, under the semi-supervised scenario, all three
patients recorded the activities they performed for six days and wrote notes on brace wear duration in a log book. In the unsupervised
setting, we continued to collect the patients’ data for additional 20 days and conducted our study with no labels. Our compliance
monitoring investigation indicated the amount of time the patient wore the brace, which matched the duration of wear prescribed by
their doctor. In this work, we introduced a priori decision tree to automatically annotate the semi-supervised data based on six
different activities of sitting, standing, walking, running, lying and climbing. We used this decision tree to verify the class labels
generated. The effectiveness of the treatment in patients with scoliosis was evaluated using three main parameters: 1) the duration for
which the brace was worn by the patient. This was measured by conducting the segmentation of the data from force sensor to evaluate
the total number of hours the brace was worn. 2) The level of tightness of the brace, by calculating the baseline force applied on the
brace. 3) Quality of the brace fit. Quality of the brace fit is important to physicians for evaluation of the brace treatment. They
recommend the brace to be worn while the patient is lying for the treatment to be effective. They also recommend the brace to be taken
off during intense physical activity like playing football. In order to determine the quality of brace fit, we classified the patient data into
six different activities of sitting, standing, walking, climbing, running and lying. We extracted the data annotated by the patient from
the logs used for creating a predictive model for activity identification. Our proposed solution consists of two stages: 1) remote sensing
design data acquisition and 2) signal processing & data mining. For the remote sensing module, we designed and fabricated a sensor
board to capture data from the patient’s body. The data mining module handles signal processing and the analytical procedure per­
formed on the captured data. To reach the quality measurement with minimum detection error, we employed a two-stage activity/
brace pressure monitoring design. In the first stage, we identified the current subject activity from 6 pre-defined classes of activities
including lying down, sitting, standing, walking, running and climbing the stairs abbreviated as LIE, SIT, SND, WLK, RUN and CLB in
our context. During the second stage, we perform activity-specific force estimation. Activity-specific refers to the concept of identifying
the activity being performed by the patient at any given time via power spectral density driven feature extraction and a priori decision
tree model for decision making. Once the model identifies the activity at any instance, we then calculate the force being applied for that
specific activity. In doing so, we provide a more accurate model to evaluate how well the brace is worn in accordance with the
prescribed regime.

4. Data collection

All three patients who volunteered for this study were under the supervision of the hospital with whom we are collaborating. The
hospital follows a regulated process, and has certain guidelines that the patients need to meet before they are allowed to participate in
the study. Some of the eligibility requirements to recruit patients for evaluating treatment compliance in AIS is that 1) they have to be
in the age range of 10–16 years, and 2) have a Cobb angle in the range of 25–40◦ . Depending on the severity of the curvature in the
spine, every patient is prescribed a custom-made brace. This treatment continues for several months. The duration of treatment is
determined by the physician and orthotist on a case-by-case basis.

4.1. Data collection scenarios

After the patients were recruited, we collected and analyzed fully pervasive data from the subjects in 1) supervised 2) semi-
supervised and 3) unsupervised experimental scenarios. The patients were asked to perform the following pre-defined activities:
LIE, SIT, SND, WLK, RUN and STR for a duration of 2 min each. This data-set was considered as the supervised data collection scenario.

3
O. Dehzangi et al. Smart Health 21 (2021) 100179

The objective of doing this was to train our model on the supervised data-set. This helped us in making a correlation of the data
collected, with the activities performed by the patient for evaluation purposes. This also proved to be a great method to test the ac­
curacy of our system. The patients then begun their treatment process and the data was continuously recorded on a secure, digital SD
card placed inside the brace. During the first 6 days of wearing the brace, the patients noted the timings of specific activities they
performed daily in a logbook provided to them. This data-set was considered as the semi-supervised data collection scenario. After the
initial 6 days of semi-supervised data collection, the patients then continued wearing the brace, and were not required to record the
activities they performed. This data-set was considered as the unsupervised data collection scenario. The patients were generally
required to wear the brace for either approximately 23 h a day, or the amount of time prescribed by the physician or orthotist. The
patients were required to conduct monthly-based visits to the Orthotist, where the data was downloaded and evaluated. The data is
sampled at 40 Sample/Sec with 10-bit resolution. This data is passed through a Low Pass filter with cut off frequency of 10Hz to filter
high frequency noise. In this work, the analysis is carried out for 6 days of semi-supervised amongst all three patients. Additional
analysis on compliance studies are carried out for 20 days of unsupervised data for the patients. The compliance study for the patients
was performed to ensure that our developed methodology is successful at establishing a proven concept for evaluating compliance of
the brace treatment in an unsupervised fashion.

5. The proposed methodology

Our proposed method for pervasive and context-aware monitoring of brace treatment includes four main modules: 1) Hardware
Architecture and 2) Data pre-processing 3) Signal processing and predictive modeling 4) Predictive Analysis, as shown in Fig. 1. It
illustrates the end-to-end system of our methodology. The inputs from the accelerometer, gyroscope and force sensor are sampled at a
sampling frequency of 40Hz. The sampled signal is passed to the data cleaning module, where the non numeric noisy data is replaced
with the average of previous value and the next values in the attribute. In the signal processing and predictive modeling system, the
sampled signal is filtered using a 10Hz low pass filter to remove high frequency noise. Features are extracted from the data and relevant
features are selected from the sequential feature-select module.The selected features are used to train the predictive model, and the
trained model is used to identify activities in the semi-supervised data scenario.

5.1. Hardware architecture

To provide a reliable and robust method for monitoring compliance in AIS patients, we developed a custom-made hardware so­
lution. The design of the system consists of a multi-modal sensor data acquisition board, a force sensor and a motion sensor. The
equipment is embedded in the brace, and data is continuously collected for analysis.
The sensor board contains an Atmega32u4, Atmel baseband data processor, a 9-axis MEMS MPU-9250 motion sensor from
InvenSense, which communicates using the I2C channel, and a custom-designed Honeywell FSB1500NSB force sensor placed at the
analog channel using 10-bit resolution A/D converter. The design also includes a micro SD card as a data logger and nRF8001 from
NordicÂo’’ as a Bluetooth low energy module. Fig. 2 displays the hardware schematic and Fig. 3c displays the sensor board that the
proposed hardware architecture is implemented in. The FSB1500NSB sensor has a range of measuring force between 0 and 15N. Since
this is not a significant range for measuring changes in force readings, we have amplified this range. We use a INA2322 CMOS TI
Instrumentation Amplifier with two resistors of values, R1=27 kΩ, and R2=87 kΩ, to amplify the numerical value of the force sensor.
Using these resistor values in equation (1), we can determine that the gain is 21.1. This is done to provide higher-density and more
precise analysis to the physician about changes in force readings. Using the gain value calculated previously, we can use that in
equation (2) to determine the output voltage of the amplifier.
( )
R2
Gain = 5 + 5 (1)
R1
( )
Vout = Vin+ − Vin− × Gain (2)

Ultimately, the maximum value we can record with the modified force sensor is 1048. Given that the maximum value the force
sensor can output is 15N, we can see that the amplification factor is: 69.867, which can be rounded to 70 for documentation purposes.
The proposed solution consists of our hardware design system embedded in a Boston-type TLSO. Fig. 3a shows the customized brace
with direction of axes of the motion sensor. The x-axis readings of the accelerometer indicate upward movement of the patient. The y-

Fig. 1. Architecture of the system.

4
O. Dehzangi et al. Smart Health 21 (2021) 100179

Fig. 2. Hardware schematic of the module.

Fig. 3. Picture of a PCB sensor board and force sensor.

5
O. Dehzangi et al. Smart Health 21 (2021) 100179

axis and z-axis readings give sideways and forward movement of the patient respectively. Fig. 3b shows the position of force sensor
inside the customized brace. Force sensor measures the force exerted by brace on the patient’s back.

6. Signal processing and predictive modeling design

The data analysis process consists of signal processing and modeling. The signal processing and predictive modeling module is
shown in Fig. 4. The input the data collected from the sensor board via Bluetooth or an SD card to the feature extraction module. The
priori decision tree uses the features to annotate the semi-supervised data automatically. The annotated data containing class labels is
used to create the predictive model. The 6-day data is classified into different activities using the predictive model.

6.1. Segmentation to determine compliance of brace treatment

To properly evaluate the effectiveness of the treatment, the duration and compliance of brace wear is to be monitored on a
consistent basis. Compliance to the brace treatment is defined as the time for which the brace is worn relative to the prescribed time
Lou et al. (2005). Compliance is important to physicians and patients as it provides important information about the patient’s ability to
wear the brace. If the patient has difficulties in wearing the brace, physicians can have a conversation and develop strategies to help
increase the compliance. With higher compliance, curve progression of the spine and chances of requiring surgery are much lower
Aulisa et al. (2014). In this paper, we design a methodology to segment the force sensor data to estimate the number of hours of brace
wear in a day. The segmentation process allows in-brace data to train the predictive model.
After data is collected from the sensor board, the force sensor data is initially filtered by the means of a 5-Hz Low-Pass filter. The
sampling rate is set to 40Hz. This initial filtering process removes high frequency noise from the force sensor data. As the patient is
consistently moving during the day, the data is non-stationary in nature. Activities such as walking, running and climbing stairs have
shown to possess a higher frequency, while standing, sitting and lying have a lower frequency. We divided the patient data into
windows of 4 s each, assuming that the signal is stationary in that window. The breathing pattern of the patient, which is captured from
the force sensor, is quasi periodic, with peaks which are about 1.5 s apart. By choosing a smaller window size of 1 s, it’s not possible to
extract the frequency content of the signal. A larger window may contain more than two different activities of the patient and will not
generate accurate results. An optimal window size of 4 s is chosen for the purposes of analysis. The window can be represented using
the equation (3).
{
0 if 0 < n <= N − 1
w[n] = (3)
1, otherwise

where N is the number of samples in a window. For a 4 s window, with 10 samples per second, the number of samples in the window is
160. For each 4 s window of the data, the average value of force sensor data is in the window of N samples is calculated using Equation
(4), where each window starts at kth sample and μ is the average value of force sensor values in that window and x[i] is the ith sample.
Each window is incremented by 1 s or 40 samples. Fig. 5 shows the details of segmentation. Markers are generated to show the

Fig. 4. Signal Processing and Predictive modeling.

6
O. Dehzangi et al. Smart Health 21 (2021) 100179

segments of data when the brace was worn.

1 N−∑
1+k
μ= x[i] (4)
N i=k

6.2. Calculation of baseline force using moving average

The force sensor measures the force exerted on the patient’s back by the brace. In this work, we estimate tightness of the brace from
the force sensor data. The force sensor data contains the patient’s breathing patterns and high frequency noise. We employ two
methods in order to calculate the baseline force exerted on the patient by the brace: 1) we use a moving average filter over each
segment of data to calculate the baseline force; the baseline force for each day is calculated using the moving average filter. This filter is
simple and fits online applications and also is effective in smoothing out the patient’s breathing patterns or the peaks of waveforms
during different activities. It computes the nth sample of the output sequence as the average of M1 + M2 + 1 samples of input sequence
around the nth sample. In the equation form, it is written as Oppenheim and Schafer (2013):

1 ∑M2
y[n] = x[n − k] (5)
M1 + M2 + 1 k=− M1

In this work, a moving average filter was applied on 10-s segments of data. Therefore, M is chosen as 400. Fig. 6 shows the force
sensor data and the baseline force calculated for the force for sitting data for day 8 for patient-1. The force sensor data for sitting is
quasi periodic and the peaks represent the breathing pattern of the patient. The moving average method of calculation of baseline force
filters out the breathing pattern of the patient.

6.3. Feature extraction

After segmentation, the in-brace data is passed to the feature extraction block. We used 13 different features for our analysis
including resultant acceleration, resultant gyroscope, force values, acceleration along x, y and z axis, gyroscope values along x, y and z
axis, mean x axis acceleration in a window, mean gyroscope vales in a window, mean pitch and number of footsteps. Movement along
y-axis or pitch represents the forward and backward movement of the brace and is a discriminative feature to differentiate lying from
sitting and standing.

6.3.1. Orientation of the brace(pitch)

Euler angles are the angles that are used to represent the orientation of the body. eul.
The roll, pitch and yaw represent the orientation of the brace with respect to x, y and z axis respectively. In the current research,
pitch has a tremendous importance as it represents if the patient is leaning forward or lying down. The pitch ranges from − 90◦ to +90◦ .
For better analysis and readability in the scatter plots, we try to normalize the angle by adding 90◦ to the pitch. Pitch can be calculated
byacc:
Ay
pitch = √̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅ (6)
(Ax )2 + (Az )2

Fig. 5. Segmentation by Average power of force. The part of data when brace is worn is indicated by markers.

7
O. Dehzangi et al. Smart Health 21 (2021) 100179

Fig. 6. Force exerted by the brace and the baseline value of the force.

This is one of the most important features as it is the angle at which the patient leans forward. This is feature is important to
differentiate climbing from walking as the patient has to lean forward a little to climb up the stairs. This feature can also help us to
differentiate between sitting and standing from lying Herring et al. (2010).

6.3.2. Resultant acceleration

If the vector A measures the acceleration in the accelerometer, the projections of A are Ax, Ay and Az respectively, The magnitude of
the acceleration can be calculated as using Equation (7) res:
√̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅
( )2
Ar = (Ax )2 + Ay + (Az )2 (7)

The resultant acceleration represents the walking, running or climbing pattern of the patient. The accelerometer signal contains
acceleration due to gravity. To calculate the acceleration of the brace and remove the effects of gravity, the mean of resultant ac­
celeration is subtracted from the resultant acceleration.
∑
n
μn = Ar [k] (8)
k=1

Af [k] = Ar [k] − μn (9)

6.3.3. Resultant gyroscope readings

If the vector G measures the orientation in the Gyroscope, the projections of G are Gx, Gy and Gz respectively, The magnitude of the
gyroscope readings can be calculated using Equation (10):
√̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅
( )2
Gr = (Gx )2 + Gy + (Gz )2 (10)

The resultant gyroscope readings represents the orientation of the brace.

6.3.4. Determination of number of footsteps using power density spectrum

The Fourier transform provides a better analysis when the signal is deterministic and is not corrupted by random noise. As the data
from the accelerometer is corrupted by random noise, we used Welch’s method to calculate the PSD as the noise due to small
movements in the brace are smoothed out. PSD is calcuated to evaluate the changes in the power of the signal with frequency. The most
dominant frequency in the periodogram gives information about activities including walking, running and climbing. Using the Welch’s
method of calculation of periodogram, we derived a unique equation to calculate the number of footsteps in each window. The
resultant acceleration Af is divided into L overlapping segments. A rectangular window w is applied to each segment. Fast Fourier
Transform is applied to the windowed data. The periodogram is applied for each windowed segment. The periodogram computed is
then averaged to compute the spectral estimate S (k).
Af (m) = Af (n + (l − 1)M) (11)

8
O. Dehzangi et al. Smart Health 21 (2021) 100179

where
n = 0, ...., N − 1 (12)

l = 1, ..., L (13)

where L represents the number of overlapping windows and N represents the number of data points in each window, (l − 1)M is the
starting point of lth window. The windowed periodogram Af (k) can be estimated using equation 14
∑
N− 1
Al (K) = Af (n)w(n)e− j2Nπ nk
(14)
n− 0

1
φl (k) = |Al (k)|2 , l = 0, ..., L (15)
NP

where Al is the FFT of the windowed segment, φl is the periodogram, and P denotes teh power of window (w(n)):

1 ∑
N− 1
P= |w(n)|
2
(16)
N n=0

The Welch’s estimate of Power Density Spectrum is given by equation (17).

Fig. 7. Figures showing the power density spectrum.

9
O. Dehzangi et al. Smart Health 21 (2021) 100179

1∑ L
S(k) = φ (k) (17)
L l=1 l

For each window, the power density spectrum can be calculated as:
Nf = fd *N (18)

Where fd is the dominant frequency and Nf is the number of footsteps in each window. Fig. 7 show the power density spectrum of
resultant acceleration with different activities. Fig. 7a shows the PSD of sitting data with the peak at 0Hz. Fig. 7b and c have the peaks
at 3.3Hz and 4.3Hz respectively.

6.4. Sequential feature selection

The acceleration is along a specific axis is important to identify the day-to-day activities of the patient. For instance, when the
patient climbs up or down the stairs, the direction of acceleration is mainly on the x-axis. When the patient walks, the acceleration
along z-axis is dominant. For stationary activities including sitting, standing or lying down, orientation of the brace or the angle of
rotation of y-axis (pitch), can be used as a discriminative feature. For lying, the normalized pitch is almost 180◦ , whereas for sitting and
standing, the normalized pitch is approximately 90◦ . The number of footsteps in a 10-s window is another discriminative feature that
we employ in this work. To improve the identification accuracy of the predictive model, we used sequential feature select methodology
to select different features which are important to identify the activities. These features are used to train the predictive model. Table 1
shows the features selected for the three patients. The features are different for each patient as the tightness and orientation of the
brace is different. The most important features for all the three patients are pitch and number of footsteps.

6.4.1. Classification
We use 10-fold cross-validation and employ a fine KNN classifier with different values of K. We also use a complex decision tree
classifier, and an Ensemble learner with Ada-boost for activity identification. The results of activity identification are shown in Table 2.

7. Experimental results and discussion

The experimental results in this study are examined in semi-supervised and unsupervised scenarios. In this section, we discuss the
results of three different phases. In the first phase, we discuss the results of sequential feature select. In the second phase, we discuss the
results related to training the predictive model, cross validation results and the activity identification results in semi-supervised
scenario. In the third phase, we discuss the experimental results of the compliance of brace treatment and the tightness of the
brace in both, semi-supervised and unsupervised settings.

7.1. Sequential feature select results and discussion

We used sequential feature selection to extract a different combinations of features for all patients, as depicted in Table 1. These
features are used for training the predictive model for each patient. We observe that the average x-axis acceleration in a window, pitch
and the number of footsteps in a window are three most important discriminative features for activity identification. The x-axis ac­
celeration represents the upward movement of the patient and is used as a discriminative feature to differentiate climbing from
walking and running. Pitch represents the orientation of the brace along y-axis. This is used as a discriminative feature to differentiate

Table 1
The features selected to identify different activities of the patient.
Patient Features Selected using Sequential Feature Select methodology

Patient-1 Acceleration along y-axis

Average x-axis acceleration in a window
Pitch
Number of footsteps
Patient-2 Acceleration along x-axis
Acceleration along y-axis
Gyroscope values along x-axis
Gyroscope values along y-axis
Average x-axis acceleration in a window
Pitch
Number of footsteps
Patient-3 Force Values
Acceleration along y-axis
Gyroscope values along z-axis
Average of resultant Gyroscope values in a window
Pitch
Number of footsteps

10
O. Dehzangi et al. Smart Health 21 (2021) 100179

Table 2
Accuracy of activity detection: Cross validation results.
KNN with K=1 kNN with k=3 kNN with k=5 Comp- lex deci- sion tree Bagg-ing Boos-ting
Accuracies

Patient-1 10-CV performance

Sitting 99% 99% 99% 99% 99% 99%
Standing 99% 99% 99% 99% 99% 99%
Walking 95% 96% 96% 99% 99% 99%
Climbing 86% 82% 79% 100% 100% 99%
Lying 99% 99% 99% 99% 99% 99%
Running 99% 99% 99% 96% 96% 90%
Overall 98% 97.8% 97.6% 100% 99.8% 99.8%
Accuracy
Patient-2 10-CV performance
Sitting 95% 95% 95% 95% 99% 99%
Standing 95% 94% 94% 89% 99% 99%
Walking 92% 90% 89% 96% 99% 99%
Climbing 90% 85% 79% 98% 99% 99%
Lying 99% 99% 99% 99% 100% 99%
Running 79% 75% 74% 95% 100% 100%
Overall 94.2% 93.5% 92.2% 94.9% 100% 100%
Accuracy
Patient-3 10-CV performance
Sitting 99% 98% 98% 98% 99% 99%
Standing 98% 98% 97% 94% 99% 99%
Walking 97% 95% 93% 96% 100% 100%
Climbing 96% 92% 91% 91% 99% 99%
Lying 99% 99% 99% 99% 99% 99%
Running 93% 93% 90% 98% 100% 100%
Overall 98.2% 96.5% 95.6% 96.2% 99.9% 99.9%
Accuracy

sitting and standing from lying.

7.2. Training and classification results

For each day, there is a 12 min window in the accelerometer and the gyroscope data, where the patient carried out activities
including sitting, standing, walking running, lying and climbing. Patient wrote the duration, time and sequence of activities in the
logbook provided. From the training data collected in the clinic, we observed that data collected from the stationary activities of
sitting, standing and lying capture the breathing pattern of the patient. The data is quasi-periodic with a frequency of about 1.5 s apart.
Frequency of resultant acceleration data is found to be highest for running. The activity detection accuracies of all three subjects using
different classifiers can be compared in Table 2. It can be noted that the highest overall accuracy achieved was that of patient 2. Their

Table 3
Classification Results for 6 days for 3 patients.
SIT SND WLK CLB LIE RUN hours of brace
Days wear

Patient-1 Classification results

1 0.9 0.1 0.3 0.2 0.0 0.0 1.8
2 1.3 0.3 0.8 0.1 0.5 0.0 3.2
3 1.9 0.7 0.6 0.0 0.6 0.1 4.2
4 2.1 0.7 0.8 0.1 0.5 0.1 4.4
5 0.9 0.4 0.4 0.1 2.2 0.2 4.4
6 2.8 0.7 0.9 0.2 4.6 0.0 5.9
Patient-2 Classification results
1 1.1 0.3 0.0 0.0 0.1 0.0 1.7
2 0.4 0.2 0.5 0.0 1.0 0.0 2.1
3 3.1 1.3 0.1 0.1 5.6 0.0 10.4
4 4.0 1.5 0.2 0.0 4.7 0.0 10.6
5 3.9 0.8 0.3 0.1 5.3 0.0 10.7
6 1.3 1.1 0.0 0.0 2.0 0.0 4.5
Patient-3 Classification results
1 3.6 3.5 0.4 0.3 0.8 0.1 9.0
2 5.3 1.5 0.1 0.0 0.6 0.0 7.7
3 3.9 0.6 0.2 0.1 0.0 0.0 5.0
4 7.1 4.1 0.6 0.2 1.6 0.2 14.1
5 1.4 1.0 0.1 0.1 4.4 0.0 7.1

11
O. Dehzangi et al. Smart Health 21 (2021) 100179

overall accuracy is 100% using bagging and boosting, with a 10-fold cross-validation methodology.
After training the predictive model model, we used Ensemble learner with Ada-boost to classify semi-supervised. The classification
results for semi-supervised data are shown in Table 3. We observe that during the entire 6 days of both, semi-supervised and unsu­
pervised data, the maximum duration of running is 12 min in an entire day for all the three patients. The patient was advised to take the
brace off during intense physical activities like playing soccer. We also observe that the duration of lying is about 5.3 h every day for
patient-2 indicating that patient-2 sleeps with brace on. The 2-min data extracted for each of activity for the first 6 days is used for
training the predictive model. We used sequential feature select to select the relevant features to train the predictive model. The cross-
validation results are as shown in Table 2. Lying shows the highest classification accuracy among all the activities as the angle of
rotation of y axis helps differentiate it from other activities. To get a better idea on which of the features are more suitable for creating a
model, we generated a confusion matrix as shown in Fig. 8. Fig. 8a, b and 8c show the confusion matrices for patients 1,2 and 3
respectively. The matrices show true positive rates for different activities.

7.3. Our results on the force estimation and compliance study

Table 3 shows the duration of brace wear in hours, while Table 4 shows the force exerted by the brace or tightness of brace wear in
Newton and the compliance in percentages on average. The relation between compliance to brace treatment and tightness with which
the brace is worn is shown in Fig. 9. We observe that initially, for the first 4 days the patient begins to become accustomed to the brace.
On day 1, they wear the brace for 2 h. From Week 2 onwards, compliance with the treatment has increased and reaches almost 50%, as
the patient is able to wear the brace for longer than 10 h a day. We also observe a gradual reduction in the force exerted by the brace,
which indicates that the brace is getting looser, which could be due to the patient’s wearing the TLSO more loosely or due to patient’s
curve reducing while wearing the TLSO. This information is quite important for the physician as a loose brace may not be able to
adequately control the spinal curvature.

7.4. Discussion on the results and the future work

Typically, patients will be seen every 4 months by the orthopedic surgeon after the beginning of the brace treatment. After the
initial in-brace x-ray, all other x-rays are taken out of the brace. i.e. the 4-month visit will usually be out of the brace. Full-length
posteroanterior spine radiograph of the patient will be obtained out of the brace to monitor for curve progression and thus, eval­
uate the success of brace treatment.
Given the observation shown in Fig. 9, it suggests that it would be necessary to adapt the brace tightness to the patients needs for
more consistent and yet acceptable pressure to the curved spine. Based on our results (Fig. 9), that might be required much earlier than
4 months. Therefore, consistently monitoring the patient’s in-brace pressures is vital for various reasons. Furthermore, the PI aims to

Fig. 8. Confusion matrix for three patients with ensemble classifiers using boosted trees with Ada Boost.

12
O. Dehzangi et al. Smart Health 21 (2021) 100179

Table 4
Results of compliance and force on the brace for 6 days of semi-supervised and 20 days of unsupervised data on average.
Force on the brace(N) compliance (%) Days Force on the brace(N) compliance (%)
Days

1 6.7 7.9 14 5.0 45.8

2 6.6 14.1 15 5.6 63.4
3 6.4 18.3 16 4.7 82.5
4 6.4 19.5 17 4.6 75.4
5 6.0 19.5 18 5.0 44.3
6 5.6 25.6 19 5.0 59.6
7 6.1 16.2 20 5.4 50.2
8 6.3 39.4 21 5.1 40.1
9 5.3 86.5 22 4.5 67.7
10 4.9 56.2 23 5.0 64.5
11 5.1 58.6 24 4.4 48.3
12 5.5 80.4 25 4.9 45.9
13 5.1 84.7 26 5.0 48.8

Fig. 9. Compliance and duration of the brace wear for 26 days.

take into account health-related quality of life (HRQoL) when assessing the effectiveness of conservative treatment of AIS with bracing
via patient feedback module as described in section 2.2. the authors propose an automated re-fitting session scheduling and evaluation
based on the addition of sensor data combined with subject reported comfort scores. Therefore, the orthotist and physician will be able
to set a criterion to schedule re-fitting on-demand and make better judgements on the ability to increase pad pressures and thereby
improve in-brace correction for the subjects. The proposed paradigm is illustrated in the Fig. 10. As described, if the in-brace tightness
level for a subjectâĂ Z’ s falls below a minimum threshold set by a physician, their in-brace comfort is high and their curve is not being
adequately corrected in the TLSO. This would indicate an opportunity to increase the pad pressures in order to further decrease the
curve magnitudes in the brace. Another example would be that the decrease in the pad pressures is an indication that the patient is not
fastening the straps to the prescribed tightness. The follow-up visit triggered by the low pressures might reveal that the patient was
experiencing some type of pain or other stress from wearing the TLSO and provide an opportunity for the orthotist or physician to
address this problem and return them to the prescribed wearing protocol. Consistently monitoring the patient’s in-brace pressures is
vital for various reasons. Furthermore, the authors aim to take into account health-related quality of life (HRQoL) when assessing the
effectiveness of conservative treatment of AIS with bracing via a simple patient feedback module on their mobile device to provide
their level of comfort, sleep quality, and pain. The authors consider general health perception, physical functioning, aesthetics, bodily
pain and social functioning and school activity. The physician will re-check the fit of the brace and make adjustments. If the physician
sets new (and perhaps tighter) in-brace pressure, the new recordings by the sensors at the re-fitting session will be considered as the
new optimum pressure for that patient.

8. Conclusion

In this work, we proposed a smart remote system to evaluate the effectiveness of brace treatment pervasively based on multimodal
sensors including, accelerometer, gyroscope, and force readings. We evaluated three main aspects of brace treatment 1) Compliance of
brace treatment by evaluating the duration of brace wear through the process of segmentation, 2) The in-brace tightness level by
calculating the baseline force exerted by the patient on the brace 3) Context aware monitoring by estimating the activities including,
LIE, SIT, SND, WLK, RUN and CLB performed daily and use the right filter in order to measure a better estimate of the in-brace tightness

13
O. Dehzangi et al. Smart Health 21 (2021) 100179

Fig. 10. Automatic re-fitting session scheduling and evaluation.

level. We investigated the quality of brace fit for 6 days of semi-supervised data for 3 patients. we achieved an overall accuracy of
99.8%, 100% and 99.9% for patients 1, 2 and 3, respectively for semi-supervised activity detection. We continued our study on
compliance and tightness for the remaining 20 days of unsupervised data. The patients were instructed to wear the brace for 2 h a day
initially, gradually increasing to 23 h a day. This was proven by our analysis as compliance was observed to increase from 20% to 65%
after 4 weeks. The brace is tightest when the patient begins the treatment. Initially the compliance is low at about 20% as the patient
takes time to get used to the brace and the number of hours of brace wear per day is very low. As they continue wearing the brace daily,
the spine gets corrected making the brace slightly looser and more comfortable to wear. The patient also gets used to the brace wear. As
the duration of brace wear or compliance varies every day, we have used the process of curve fitting in our graph to get a better
understanding of the compliance. Our analysis also showed that the force exerted by the brace decreased by 33% after 4 weeks. We aim
to utilize this interesting observation for automatic scheduling of re-fitting sessions to improve the effectiveness of the brace treatment
by conducting more consistent pressure throughout the long treatment paradigm.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to
influence the work reported in this paper.

Acknowledgments

The authors would like to thank the Orthotics and Prosthetics Center at the University of Michigan for their invaluable cooperation
in the design and development of the braces and their input and help with embedding sensors in the smart braces.

References

Asher, M. A., & Burton, D. C. (2006). Adolescent idiopathic scoliosis: Natural history and long term treatment effects. Scoliosis, 1, 2.
Aulisa, A. G., Giordano, M., Falciglia, F., Marzetti, E., Poscia, A., & Guzzanti, V. (2014). Correlation between compliance and brace treatment in juvenile and
adolescent idiopathic scoliosis: Sosort 2014 award winner. Scoliosis, 9, 6.
Bao, L., & Intille, S. S. (2004). Activity recognition from user-annotated acceleration data. In International conference on pervasive computing (pp. 1–17). Springer.
Capela, N. A., Lemaire, E. D., & Baddour, N. (2015). Improving classification of sit, stand, and lie in a smartphone human activity recognition system. In Medical
measurements and applications (MeMeA), 2015 IEEE international symposium on (pp. 473–478). IEEE.
Dolan, L. A., Wright, J. G., & Weinstein, S. L. (2014). Effects of bracing in adolescents with idiopathic scoliosis. New England Journal of Medicine, 370, 681.

14
O. Dehzangi et al. Smart Health 21 (2021) 100179

Ermes, M., Pärkkä, J., Mäntyjärvi, J., & Korhonen, I. (2008). Detection of daily activities and sports with wearable sensors in controlled and uncontrolled conditions.
IEEE Transactions on Information Technology in Biomedicine, 12, 20–26.
Herring, J. A., Katz, D. E., Browne, R. H., Kelly, D. M., & Birch, J. G. (2010). Brace wear control of curve progression in adolescent idiopathic scoliosis. J Bone Joint
Surg Am, 92, 2616–2617.
Karol, L. A., Virostek, D., Felton, K., & Wheeler, L. (2016). Effect of compliance counseling on brace use and success in patients with adolescent idiopathic scoliosis. J
Bone Joint Surg Am, 98, 9–14.
Lou, E., Hill, D. L., Raso, J., Moreau, M. J., & Mahood, J. K. (2005). Smart orthosis for the treatment of adolescent idiopathic scoliosis. Medical, & Biological Engineering
& Computing, 43, 746–750.
Morton, A., Riddle, R., Buchanan, R., Katz, D., & Birch, J. (2008). Accuracy in the prediction and estimation of adherence to bracewear before and during treatment of
adolescent idiopathic scoliosis. Journal of Pediatric Orthopaedics, 28, 336–341.
Oppenheim, A., & Schafer, R. (2013). Discrete-time Signal Processing. Always learning. Pearson. https://books.google.com/books?id=LeQpnwEACAAJ.
Pereira, J. D., da Silva e Silva, F. J., Coutinho, L. R., de Tácio Pereira Gomes, B., & Endler, M. (2016). A movement activity recognition pervasive system for patient
monitoring in ambient assisted living. In Proceedings of the 31st annual ACM symposium on applied computing (pp. 155–161). ACM.
Rahman, T., Bowen, J. R., Takemitsu, M., & Scott, C. (2005). The association between brace compliance and outcome for patients with idiopathic scoliosis. Journal of
Pediatric Orthopaedics, 25, 420–422.
Ribeiro Filho, J. D. P., e Silva, F. J.d. S., Coutinho, L. R., & Gomes, B.d. T. P. (2016). Mhars: A mobile system for human activity recognition and inference of health
situations in ambient assisted living. Journal of Applied Computing Research, 5, 44–58.
Sapountzi-Krepia, D., Psychogiou, M., Peterson, D., Zafiri, V., Iordanopoulou, E., Michailidou, F., & Christodoulou, A. (2006). The experience of brace treatment in
chil- dren/adolescents with scoliosis. Scoliosis, 1, 8.
Weinstein, S. L., Dolan, L. A., Wright, J. G., & Dobbs, M. B. (2013). Effects of bracing in adolescents with idiopathic scoliosis. New England Journal of Medicine, 369,
1512–1521.

Further reading

A guide to using imu in embedded applications. URL: http://www.starlino.com/imu_guide.html.

Arduino imu pitch and roll from an accelerometer. URL: https://theccontinuum.com/2012/09/24/arduino-imu-pitch-roll-from-accelerometer/.
Euler angles. URL: https://en.wikipedia.org/wiki/Euler_angles.

15