Journal of Biomedical Informatics: X 6–7 (2020) 100075

Contents lists available at ScienceDirect

Journal of Biomedical Informatics: X

journal homepage: www.journals.elsevier.com/journal-of-biomedical-informatics-x

Prediction and pattern analysis of medication refill adherence through T

electronic health records and dispensation data
Alexander Galozy , Slawomir Nowaczyk
⁎

Center for Applied Intelligent Systems Research, 30118 Halmstad, Sweden

ARTICLE INFO ABSTRACT

Keywords: Background and purpose: Low adherence to medication in chronic disease patients leads to increased morbidity,
Medication refill adherence mortality, and healthcare costs. The widespread adoption of electronic prescription and dispensation records
Electronic health records allows a more comprehensive overview of medication utilization. In combination with electronic health records
Simulation (EHR), such data provides new opportunities for identifying patients at risk of nonadherence and provide more
Prediction
targeted and effective interventions. The purpose of this article is to study the predictability of medication
Refill patterns
adherence for a cohort of hypertensive patients, focusing on healthcare utilization factors under various pre-
dictive scenarios. Furthermore, we discover common proportion of days covered patterns (PDC-patterns) for
patients with index prescriptions and simulate medication-taking behaviours that might explain observed pat-
terns.
Procedures: We predict refill adherence focusing on factors of healthcare utilization, such as visits, prescription
information and demographics of patient and prescriber. We train models with machine learning algorithms,
using four different data splits: stratified random, patient, temporal forward prediction with and without index
patients. We extract frequent, two-year long PDC-patterns using K-means clustering and investigate five simple
models of medication-taking that can generate such PDC-patterns.
Findings: Model performance varies between data splits (AUC test set: 0.77–0.89). Including historical in-
formation increases the performance slightly in most cases (approx. 1–2% absolute AUC uplift). Models show
low predictive performance (AUC test set: 0.56–0.66) on index-prescriptions and patients with sudden drops in
PDC (Recall: 0.58–0.63). We find 21 distinct two-year PDC-patterns, ranging from good adherence to inter-
mittent gaps and early discontinuation in the first or second year. Simulations show that observed PDC-patterns
can only be explained by specific medication consumption behaviours.
Conclusions: Prediction models developed using EHR exhibit bias towards patients with high healthcare utili-
zation. Even though actual medication-taking is not observable, consumption patterns may not be as arbitrary,
provided that medication refilling and consumption is linked.

1. Introduction Significant efforts have been undertaken to develop interventions

Medication nonadherence, or the failure of patients to take their
medication as prescribed, is a major public health concern that con-
tributes to increased morbidity, mortality and health care costs [10]. It
is estimated that about 30–50% of patients with chronic diseases fail to
adhere to their medication regiment as recommended by their physi-
cian, leading to poor clinical outcomes, unnecessary health care costs
due to increased hospitalizations and ineffective utilization of medica-
tion resources [6].
targeted at nonadherent patients [17]. The great challenge is to identify
patients at risk as early as possible so that these interventions can be
most effective. For many cardiovascular diseases, timeliness and ef-
fectiveness of pharmacological treatment are tightly linked, especially
for secondary or tertiary prevention [16].
In order to identify patients that are nonadherent, the level of ad-
herence needs to be quantified, but accurate and cost-effective mea-
surement of adherence is a key challenge. Direct measures, such as
measurement of metabolite concentration in body fluids of a particular

⁎
Corresponding author.
E-mail address: alexander.galozy@hh.se (A. Galozy).
URL: http://www.islab.hh.se/mediawiki/Alexander_Galozy (A. Galozy).

https://doi.org/10.1016/j.yjbinx.2020.100075

Available online 13 June 2020

2590-177X/ © 2020 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY license
(http://creativecommons.org/licenses/BY/4.0/).
A. Galozy and S. Nowaczyk
drug, provide the highest accuracy but place a high burden on patients.
Less accurate and rigorous measures include the assessment of the
clinician via patient interviews and self-report through patient-kept
diaries [21]. Indirect measures approximate medication adherence
through medication refill adherence by utilizing proxy information
such as pharmacy dispensation data or insurance claims data, providing
a low cost, low burden solution, with the caveat that actually medica-
tion consumption remains unknown. Nevertheless, new tools created by
the wealth of information through the advancing digitization of
healthcare, provide fertile ground for valid and cost-effective analysis
of refilling behaviour as well as timing of drug exposure [3,9].
Many different ways of computing medication refill adherence,
hence forth called refill adherence, using secondary databases have
been developed, with Proportion of Days Covered (PDC) and
Medication Possession Ratio (MPR) being the most popular ones. Those
measures expresses the acquired number of doses over a set period as a
percentage, ordinarily in the range between 0% and 100%. Even though
many measures exists, PDC is being recommended by the Pharmacy
Quality Alliance (PQA) as the defacto standard measure.1
Previous studies on predicting refill adherence utilizing socio-
demographic factors, clinical factors, or purchasing information had
limited success, with models exhibiting only low discriminability, often
with AUCs < 0.7 [30,18,13,19]. Although these efforts have had only
marginal success in developing accurate predictive models, they pro-
vided valuable insights into the importance of analyzing patients’ re-
filling behaviours and health care utilization patterns. Several studies
find that initial and past patterns of medication refilling can sig-
nificantly improve model performance, providing the majority of the
models’ predictive accuracy [12,13,22,20].
As mentioned, a limiting factor of refill adherence analysis is that
patients that refill their medication do not necessarily take them as
prescribed, but there exists some evidence to suggest a link between
refilling and medication-taking [14]. Analyzing medication refill pat-
terns might provide insight into the possible consumption patterns of
medication, especially early on in pharmacological treatment where
long-term effects of, for example, hypertension can be mitigated.
This study was part of a project providing personalized interven-
tions and support for patients struggling with adherence to pharma-
cological treatment.2 Therefore, we first analyze refill adherence of a
large and diverse cohort of patients diagnosed with and treated for
hypertension (ICD i10-) under various predictive scenarios to get a
general idea about the predictability of refill adherence and potential
model or data bias, as we will show, using data from a comprehensive
EHR system. We focus specifically on healthcare utilization factors,
such as visit frequency/type, past prescription information, past PDC,
and a limited number of demographic variables of both patient and
prescriber.
Secondly, we discover common PDC-patterns via unsupervised
clustering of a sub-cohort of patients that have been prescribed anti-
hypertensive medication for the first time. This sub-cohort constitutes a
more typical group of patients that might be target by interventions in
the project mentioned above, where early intervention can have argu-
ably the highest impact on long term positive health outcomes.
Furthermore, under the assumption of a link between refilling and
medication-taking behaviour, we construct simple simulation models,
generating diverse PDC-patterns and medication consumption patterns
that could explain refill behaviour we observe in the patient sub-cohort.
1
https://www.pqaalliance.org/adherence-measures - last accessed: 2020-04-
13.
2
https://www.hh.se/english/research/research-environments/embedded-
and-intelligent-systems-eis/technology-area-aware-intelligent-systems/imeda-
digital-support-for-high-blood-pressure.html - last accessed 2020-04-13.
2
Journal of Biomedical Informatics: X 6–7 (2020) 100075
2. Methods
2.1. Data source
We analyzed data from the Region Halland (RH) data warehouse
containing detailed patient records from public primary, secondary,
and specialist care facilities in Halland, Sweden. Data from private care
providers through a data agreement with RH are available, providing a
comprehensive view of the patients’ encounters with the regions’ health
care system. Clinical data include lab results, radiology diagnostics,
diagnoses, waveform data, doctors’ notes, and performed medical
procedures. Through data integration efforts with Swedish pharmacies,
pharmacy dispensation data is available. As of October 2018, records
from 545,652 patients (male 49.2%/ female 50.8%), totaling 7,548,301
visits, from 29 care centers (24 primary care, 3 hospitals, 2 emergency
care) are included. The data is pseudonymized with each patient’s 10-
digit Swedish Personal ID being replaced with a unique randomized ID
that allows tracking through the healthcare system without exposing
the identity of the patient. All identifying information such as: name,
address, date of birth and telephone numbers are removed. The plat-
form has been designed to conform to Swedish and EU laws and reg-
ulations on data privacy and confidentiality [4].
2.2. Cohort
The cohort consists of patients between the ages of 18 and 90 with
an essential hypertension diagnosis (ICD10 code i10-) at one point
between the dates 1st November 2014 and 31st July 2018. We assume
the full medical history of the patients is known. Therefore we did not
consider further exclusion criteria on patient-level, such as specific
comorbidities or excluding patients with hospital visits as has been
common in previous studies that lack the same completeness of the
EHR. There is a possibility that patients visit hospitals or primary care
centres outside RH and may receive prescriptions that are subsequently
missing in the records, but we do not expect this to alter results sig-
nificantly. Dispensation records are available from all dispensaries in
Sweden; this mitigates the possibility of underestimation of adherence.
We predict refill adherence for one-year prescriptions of medication
commonly prescribed for antihypertensive treatment. Brand names and
generic names of the prescribed and dispensed medication are avail-
able. However, due to missing joining keys between prescription and
dispensation systems and the possibility that patients might change
brand names within a prescription, we opted to use the Anatomical
Therapeutic Chemical Classification System (ATC) code to allow for
simpler matching of dispensations and prescriptions. As part of the
project mentioned above, we primarily consider medication with ATC
codes that are commonly prescribed for patients with high blood
pressure: C02- (antihypertensives), C03- (diuretics), C07- (beta-
blocking agents), C08- (calcium channel blockers) and C09- (Agents
acting on the renin-angiotensin system).
2.3. Data preprocessing
Before matching dispensations to prescriptions, we removed dupli-
cations and adjust prescription periods. The exact procedure is detailed
in an earlier published article but briefly summarized here for com-
pleteness [2].
In all cases, adherence is evaluated in the prescription period, the
time frame between the prescription start and end date. Dispensation
data is available between 28th May 2012 up until 31st July 2019, while
prescription data is available from 1st January 2010. To avoid missing
dispensations, which would lead to underestimation of adherence, we
exclude patients with index prescriptions before the 28th May 2012.
Average dispensation rates approx. take until 1st November 2013 to
fully settle. We exclude patients that have index prescriptions until this
date to reduce the chance of missed dispensations further. In case
A. Galozy and S. Nowaczyk Journal of Biomedical Informatics: X 6–7 (2020) 100075

harder to measure accurately, especially since such information is not

contain in EHRs. Such individual factors are expected to be somewhat
correlated through patients healthcare utilization, which we included in
the form of visits. The predictors, called baseline predictors in this
study, include demographic information of the patients (age/gender) at
the time of prescription, prescriber age, aggregate predictors (pholy-
pharmacy, ATC-code diversity, years under treatment, number of pre-
Fig. 1. Filtering of patients to ensure completeness of the data set. Patients with vious prescriptions) of current and past prescriptions, as well as visit
index prescriptions exceeding the 1st November 2014 and end dates earlier frequency and visit types, i.e., outpatient, inpatient and emergency care
than the 31st July 2019 are included in the analysis. visits. While data on blood pressure measurement and BMI are avail-
able, their input is not done in a systematic fashion and often only
available in free text format that does not follow standardized input
patients received treatment outside of RH, we allow for a run-out period
patterns. Such information suffers from low data quality with often
of one year to ensure earlier dispensations have not been missed. The
missing entries or wrong input that does not allow conclusions about
final cohort constitutes patients with index prescriptions (first record in
the correct values for the majority of entries. Therefore we omit such
the system) exceeding the 1st November 2014 and with prescription
predictors from consideration.
end dates not exceeding the 31st July 2019. Fig. 1 illustrates the pro-
For comparative purposes, we provide the algorithms with the
cess.
baseline predictors of the five most recent prescriptions. Additionally,
Overlapping prescriptions for the same drug had their prescription
we add the days between start dates of prescriptions and proportion of
time-frames adjusted, i.e., older prescriptions terminate when a new
days covered of each recent prescription (expect the current one).
one is issued. We removed duplicate prescriptions issued on the same
Missing past information, such the case for patients with less than five
date for the same ATC-Code retaining the latest prescription.
prescriptions, is imputed with placeholder values, in our case with 1.
We imputed missing prescription end dates (6.35% of all prescrip-
The predictors and their descriptions are shown in Table 1. We hy-
tions) with a standard prescription length of one year. One year pre-
pothesize that additional information from previous prescriptions helps
scriptions are common for chronic disease medication, where patients
in predicting future refill adherence. In summary, we investigate two
have an annual checkup to renew their prescriptions.
general scenarios: Current baseline predictors of each prescription and
baseline predictors of the five most recent prescriptions. Additionally,
2.4. Measuring refill adherence
we investigate the influence of several data splitting strategies on pre-
dictive performance.
We compute adherence for each prescription using the proportion of
days covered (PDC) measure. PDC is defined as:
2.5.2. Training, validation & test sets
#gap days We utilize different data splitting strategies, evaluating model per-
PDC = 1
#days within prescription (1) formance under various predictive scenarios. Training, validation and
test sets contain similar numbers of prescriptions for all splitting stra-
where #gaps days refers to the number of days where medication is
tegies. We use the validation set for hyper-parameter tuning of the
theoretically missing, usually when refills are not made on time and
models. If not mentioned otherwise, we split the data set into 70%
#days within prescription are the number of days the prescription is
training, 15% validation and 15% test. We ensure a similar class bal-
valid. Prescription lengths are usually one year but may vary depending
ance between data sets.
on early terminations or extensions. Each prescription can have mul-
Stratified random split. Training data is sampled randomly from
tiple medications, in which case we predict adherence for each medi-
the data, stratified by class distribution. This strategy assumes that
cation separately. Furthermore, we include supply carryover in the PDC
samples in both training and test sets are independent and identically
computation, if available from previous prescriptions.
distributed (iid), i.e., the is no change in distribution between old and
new data. This strategy is one of the most commonly applied, especially
2.5. Predicting refill adherence
in combination with random k-fold cross-validation schemes, to esti-
mate the models’ performance on unseen data.
In this section, we describe in detail how we designed the prediction
Splitting by patient. The patients in the training, test and valida-
study, what prediction scenarios we investigated, and how we evaluate
tion set are disjoint, i.e., the training set does not contain data from
model performance.
patients in the test set or validation set. The same separation of patients
applies to the test and validation set. Each data set contains unique
2.5.1. Classification label & predictors
patents, mimicking the situation where new patients need to be clas-
An 80% threshold is commonly used in adherence and prediction
sified, with the assumption that refilling behaviour can be inferred from
studies for arbitrary or historical reasons, above which it is assumed
the behaviour of other patients.
that pharmacological treatment is effective [5,7]. Nevertheless, the
We investigate two different approaches for splitting the data based
PQA advocates for such a threshold for chronic disease therapy.3 To
on time.
allow for comparisons of our results to previous studies, we dichot-
Temporal split: most recent prescription per patient. For each
omized the computed adherence values using an 80% threshold. Pa-
patient, we reserve the most recent prescription for the test set. The
tients at or above 80% are considered adherent to a particular pre-
validation set contains the second most recent prescription. If patients
scription and nonadherent otherwise.
only have one recent prescription, they are included in the training set,
As per the definition of the WHO, adherence is determined by the
i.e., the test and validation sets do not contain patients that only have
interplay of a multitude of factors. In our prediction study, we mainly
one prescription. Therefore, we expect higher model performance due
focus on patient-related factors and demographic factors. While de-
to lower refill adherence variability for patients with more extended
mographic factors are available explicitly, patient-related factors or
treatment history.
behavioural factors such as “being busy” or “treatment satisfaction” are
Temporal split: most recent prescription in the record. We re-
serve the most recent prescription (independent of patient) for the test
3
https://www.pqaalliance.org/adherence-measures - last accessed: 2020-04- set. This scenario is likely to occur in practice after model deployment,
13. where old data is used building the model. Validation and test sets can

3
A. Galozy and S. Nowaczyk Journal of Biomedical Informatics: X 6–7 (2020) 100075

Table 1 restricting the analysis to where a patient is nonadherent based on

Predictors and their description. For the scenario considering the last five aggregating measures which a set arbitrary threshold, does not provide
prescription (history), two additional predictors are included. information about specific patterns of adherence that might be clini-
Predictor Description cally relevant, leaving behind an additional source of information to
characterize adherence [8]. Analyzing how patients dispense medica-
PatientAge Patient Age at Prescription tion and reasoning about possible medication-taking behaviours pro-
PatientGender Patient Gender
vides additional information that could be used to decide what and
NumPrepYear Number of prescription in particular year
NumDrugClass Number of different drug classes when intervention might be appropriate.
DrugYear Years since first time prescription Given that patients can arbitrarily consume medication, PDC-pat-
PresQuantity Dose prescribed terns may not correspond to actual medication-taking limiting the use
NumPolypharm Number of concurrent prescriptions
of refill adherence as a proxy for actual adherence. Nonetheless, it is
OverSupply Assumed available supply for prescription
PrescriberAge Age of Prescriber at time of prescription
reasonable to assume and corroborated by previous research [14], that
NumOutvisits Number of outpatient visits patients that are nonadherent to medication, e.g., discontinue treat-
NumInvisits Number of inpatient visits ment or do not take them as instructed, eventually stop refilling or tend
NumEMvisits Number of emergency visits to refill their medication irregularly. Provided we make some as-
DiffOutvisits Number of outpatient visits since last prescription
sumptions about when patients refill their medication, and how this
DiffInvisits Number of inpatient visits since last prescription
DiffEMvisits Number of emergency visits since last prescription refilling might relate to actual medication-taking, we can construct si-
mulation models that explain patterns we observe in the dispensation
Additional predictors when considering history
DistToPrev Distance (days) to previous prescription records.
PDC Adherence to previous prescription The focus of our simulation study is on patients that anti-
hypertensive medication for the first time and conduct simulations of
medication-taking that could explain the observed PDC-patterns.
contain new patients and the model is therefore forced to extrapolate Previous studies highlight the importance of medication management
adherence for new and old patients alike. in the early stages of hypertension, retarding, or preventing the onset of
organ damage due to high blood pressure [29,1]. Therefore it is of in-
2.5.3. Machine learning algorithms & evaluation terest to understand how patients take the medication in the early
We train our prediction models using four popular machine learning stages of hypertension and allow for early intervention if need be.
algorithms: Random Forest (RF), Logistic Regression(LR), Gradient
Boosting Trees (GB) and k-nearest neighbour (KNN). Each of the algo- 2.6.2. Sub-selected cohort
rithms exposes parameters that we optimize on the validation sets for We analyze adherence patterns of a sub-selected group of patients
each predictive scenario. There exists a class imbalance due to the from the prediction study, having hypertension diagnosis and receiving
majority of patients (approx. 80%) being adherent to medication. This antihypertensive monotherapy (single or combination drug) for the first
imbalance is accounted for by stronger penalization of model mistakes time. We remove patients with severe health-states or patients which
on the minority class during training. The penalty is inversely propor- are outside the typical age group for ongoing antihypertensive treat-
tional to class frequency, i.e. mistakes on the minority class are pena- ment, as they might exhibit atypical prescription and PDC-patterns not
lized about five times higher during training, shifting prediction per- numerous enough to extract general PDC-patterns. Patients are between
formance in favour of the less frequent class. the ages of 40–70 years at treatment initiation do not have a history of
We measure model performance primarily with the area under the stroke, myocardial infarction, heart failure or diabetes. Patients with
receiver operating characteristic curve (ROC-AUC or AUC for short), pregnancy-induced hypertension one year before treatment initiation
also know as the C-statistic, as well as precision and recall. Precision is are excluded. We exclude patients that are being treated for psychiatric
defined as: disorders. All selected patients receive monotherapy with a dose of one
TP pill a day. The sub-selected cohort constitutes 6305 patients (33.9% of
PRECISION =
(2) the prediction dataset).
TP + FP
and recall is defined as: 2.6.3. PDC-pattern clustering
TP We cluster PDC-patterns of prescription for the first two years of
RECALL = pharmacological treatment. A fully dispensed prescription constitutes
TP + FN (3)
400 pills which lasts for 400 days. We therefore analyse the first
with abbreviations: True positive (TP), false positive (FP), true negative 800 days constituting two years of treatment. The PDC-patterns are
(TN) and false negative (FN). We measure precision and recall at the represented as binary vectors of length 800, indicating if a particular
decision threshold of 0.5 for all models. day is covered (1) or not covered (0) by medication. The first day of
treatment starts with the index-prescription and varies for each patient.
2.6. PDC-pattern analysis: A simulation study For clustering, we use the K-means algorithm [26] with a fixed number
of K = 100 clusters and the euclidean distance measure. Given the high
In this section, we describe why and how we find common patterns dimensionality of the binary vectors, other clustering algorithms such
of refilling. We describe the simulation models of medication-taking as DBSCAN [11], K-shape [27] with other distance-metrics such as the
and how we matched the simulation results to the observed patterns. Jaccard distance and dynamic time warping have been investigated but
did not result in different or subjectively better clustering results. We
2.6.1. Why are refill patterns interesting? chose the number of clusters empirically, not based on metrics such as
Predicting medication adherence would allow early intervention to the silhouette score, which were quite uninformative for deciding the
mitigate the impact of nonadherence on the patients’ health. As we will optimal number of clusters. We did not see a significant change in the
show, due to significant sampling bias in EHRs, i.e. patients with high diversity of patterns going above K = 100. After clustering, we ag-
healthcare utilization are overrepresented, prediction models might be glomerated the set of 100 cluster centroids into a smaller set based on
unreliable on patients with low refill adherence. Furthermore, their shape, and extracted a representative centroid identified by

4
A. Galozy and S. Nowaczyk
manual inspection, reducing the number of cluster centroids to 21 two-
year PDC-patterns.
2.6.4. Simulation model for medication-taking
We employ a simple model to simulate medication-taking. The
primary mechanism determining medication-taking in this model is
“forgetting” and the concept of “motivation”, i.e., the desire of the
patient to abide by their prescribed treatment regimen. We model
motivation by simple mathematical equations and investigate five basic
models.
The general model of medication consumption is governed by
combining the outcomes of two iid Bernoulli trials:
P (Xc = 1) = P (X1 = 1) P (X2 = 1) (4)
With Xc being a binary random variable indicating medication con-
sumption. The binary random variables X1 and X2 are distributed ac-
cording to:
X1 ~B (1, M ), X2 ~B (1, 1 cfprob)
With M (0, 1) being the level of motivation expressed as a probability
and cfrob (0, 1) being the probability of forgetting to take medication.
This model of medication-taking applies to all motivation models.
For the constant model, motivation does not change over time:
M = cconst (5)
where 0 cconst 1. The constant model can generate a variety of PDC-
patterns associated with patients having low conscientiousness in
medication-taking, i.e., patients are somewhat willing to take mediation
but forget at random times, intentionally or unintentionally.
In the seasonal model, motivation does decrease over time(ex-
ponentially) but increases(suddenly) when the time for a follow-up
prescriptions arises, i.e., patients might feel pressure to fulfill their re-
gimen before visiting their doctor.
(t mod cseasonality)·cdecline
M (t ) = e cseasonality (6)
where t is the current simulation iteration and cseasonality defining the
time (in days) between sudden changes of motivation.
For the sigmoidal model, motivation can change somewhat
abruptly, either increasing, decreasing or having a short period of low
or high motivation. Patients might discontinue medication on their own
for some time with a re-uptake of regimen later on. We define a base
model:
t + coffset + cgap
m (t , coffset , cgap) = 1/(1 + e cdecline ) (7)
With coffset defining the time at which patient motivation changes. cgap
defines the time between lowering or increasing motivation levels. We
then construct several motivational patterns:
m (t , coffset , 0), for n = 1
1 m (t , coffset , 0), for n = 0
M (t , n) =
[1 m (t , coffset , 0)]+
m (t , coffset , cgap), for n = 1 centroids directly, we chose one simulation pattern that is closest to the
For the exponential model motivation decreases exponentially over
time. Refills influence the rate of decrease, i.e., refills reset the rate of
motivation loss. The exponential model is defined recursively.
t cdisp
M (t ) = M (t 1)· e cdecline·citer (8)
where cdisp measures the number of days since the last pickup and citer
defines the total number of iteration, i.e., days (here 800).
The sinusoidal model generates recurring periods of high and low
motivation. Medication-taking characterized by stretches of medication
adherence and nonadherence. The sinusoidal model is defined as:
5
Journal of Biomedical Informatics: X 6–7 (2020) 100075
(t + coffset )
M (t ) = 1 + c yinterval· cos 1
cdecline (9)
With c yinterval defining the amplitude between motivation “highs” and
“lows”, e.g. for c yinterval = 0.5 motivation oscillates between 1 and 0.
As mentioned earlier, real patients can consume medication arbi-
trarily and there does not need to be any link between refill patterns
and actual medication use. In the absence of any direct information
about how a particular patient is consuming medication, it is reason-
able to assume that patients that are motivated to take their medication
are also more likely to refill their medication regularly. Provided that
this assumption does mirror reality, we can reduce the number of all
possible medication consumption patterns given particular refilling
behaviour.
Therefore, for our simulated patients, we assume that there exists a
link between available medication supply and refilling patterns. For
simplicity, the probability of refilling is only dependent on the supply
currently available to the patient, i.e., the patient delays refilling if
supply is still available. The probability of pickup Xp is computed as
follows:
e s
P (Xp = 1|s , t ) = cdisp (t )
1+e s (10)
where cdisp (t ) is the indicator function defined as:
1 if t cdisp
cdisp (t )
0 if t < cdisp
Due to limits in dispensation frequency at Swedish pharmacies, (real)
patients are not allowed to dispense all packages of medication at one
time, needing to wait before receiving a new package. We model this
for our simulated patients with the dispensation threshold cdisp that
limits the dispensation frequency. The probability of pickup is strongly
coupled with available supply, that is to say, patients pick up medica-
tion if they nearly exhausted their current supply.
Simulated patients pick up 100 days of supply per dispensation,
constituting a standard supply amount for antihypertensive medication
in Sweden. We take into account the possibility that patients might
have 100 days supply available at the beginning of the prescription
without a registered dispensation. Fig. 2 illustrates simulated medica-
tion-taking, refilling and PDC-patterns.
2.6.5. Pattern generation & matching
We run a Monte Carlo Simulation [15] for 800 days (approx. two
years) of medication-taking for each of the motivation models. We run
each simulation with a variety of different parameters to ensure the
diversity of PDC-patterns. The list of parameters for the simulation
models is shown in Table 2. As a result of convergence analysis, see
Fig. 3, each model is run 101 times for each parameter combination and
the median PDC-pattern computed. We match the simulated PDC-pat-
terns and PDC-patterns found by cluster analysis in a step by step
fashion: First, instead of matching individual simulation runs to cluster
median of the 101 runs. Given the stochastic nature of each model,
single runs may not accurately reflect expected model behavior for a set
of parameters. To mitigate the impact of output variability, we chose a
representative pattern that is close to the median output of the model.
We do this by finding the simulation pattern that minimizes the sum of
squared difference between individual PDC-pattern and the median
PDC-pattern:
citer
j, k
Xrep = arg min (Xi j, k (t )sim Xi j, k (t )median ) 2
1 i 101 t=1 (11)
This individual pattern is the representative for both model k and
A. Galozy and S. Nowaczyk Journal of Biomedical Informatics: X 6–7 (2020) 100075

Fig. 2. Medication-taking (Drug taken), refill (Dispensation) and PDC-patterns (Day covered) generated by the proposed simulation models. The models are capable
of generating a large variety of medication consumption and corresponding PDC-patterns solely based on current motivation levels and forgetting.

specific set of model parameters j.

Secondly, we repeat the procedure for cluster centroids found
through cluster analysis. We find the nearest neighbor representative
simulation pattern Xrep
j, k
for cluster centroid C p , that minimizes the sum
of squared difference:
citer
k
Xrep centroid = arg min (X j, k (t )rep C p (t )) 2
j t=1 (12)

In summary: We simulate each model several times to account for

stochastic variability for the same model parameters. We then find a
representative pattern that is close to the median pattern of the 101
simulation runs using Eq. (11). We do this to account for the fact that
the median pattern does not necessarily reflect the true shape of any of
the simulated patterns. Finally, we find the representative patterns (of
each model) that most closely matches the cluster centroid C p for all 21
pattern clusters, using Eq. (12).
Fig. 3. Maximum difference between means for each iteration. The figure
Ethical approval for this study: ethics committee in Lund, D.nr. shows averages and 95% CI between all simulation models. We observe the
2018/294. convergence of the mean after about 75 iterations. we chose 101 runs for each
simulation to ensure good approximation of the mean pattern.

Table 2
Model parameters, their description and ranges for the simulation experiments.
Parameter Description Range

cdisp Dispensation frequency threshold in days 60 (fixed)

motivtype Motivation type constant, seasonal, sigmoidal, exponential, sinusoidal
initsupply Initial supply available [0, 100]
cconst Motivation parameter of constant model [0.95, 0.75, 0.5, 0.2]
cfprob Probability of forgetting [0.8, 0.6, 0.4, 0.2, 0.1, 0.0]
coffset Model specific parameter. Timeliness of motivation change [540, 450, 360, 270, 180, 90, 0]
cgap Sigmoidal model parameter. Defines the gap (in days) between motivation changes [200, 100, 50, 30]
cdecline Model specific parameter. Frequency or magnitude of motivation change [300, 270, 200, 100, 50, 40, 30, 20, 10, 5, 3, 2, 1, 0.7, 0.5, 0.2, 0.1]
cseasonality Seasonal model parameter. Rapid increase in motivation at fixed intervals [400, 365]

6
A. Galozy and S. Nowaczyk Journal of Biomedical Informatics: X 6–7 (2020) 100075

Table 3
Descriptive statistics of predictors between training, validation and test sets of different splitting strategies. Long predictor labels are shown for easier reading. Mean
and standard deviating shown.
Splitting strategy Random Patient

Data set Training Validation Test Training Validation Test

Predictors mean std mean std mean std mean std mean std mean std

Patient Age at prescription 67.1 12.3 67.1 12.4 66.9 12.4 67.1 12.3 66.9 12.4 67.3 12.6
Patient Gender∗ 0.514 0.5 0.523 0.5 0.512 0.5 0.508 0.5 0.534 0.499 0.529 0.499
Prescriber Age 41.7 11.0 41.8 11.0 41.8 11.1 41.6 11.0 42.2 11.3 41.8 10.9
Prescription quantity 1.06 0.329 1.06 0.307 1.06 0.31 1.06 0.309 1.06 0.361 1.06 0.343
Num. concurrent drugs 1.81 0.911 1.83 0.929 1.81 0.913 1.82 0.917 1.82 0.91 1.82 0.902
Num. prescriptions in year 1.02 1.48 1.02 1.48 0.994 1.48 1.02 1.49 0.995 1.45 1.01 1.47
Num. different drug classes 1.75 1.5 1.76 1.53 1.75 1.51 1.76 1.52 1.74 1.49 1.74 1.47
Years since index date of drug 1.72 0.917 1.73 0.927 1.71 0.912 1.73 0.921 1.72 0.92 1.7 0.9
Oversupply 48.9 108.0 50.8 113.0 50.4 116.0 50.1 109.0 46.1 110.0 49.3 112.0
Num. primary care visits 43.1 53.7 42.4 54.2 42.6 51.7 43.1 54.3 40.6 50.3 44.3 52.7
Num. hospital Visits 1.04 1.84 1.05 1.94 1.01 1.79 1.02 1.85 1.05 1.81 1.13 1.85
Num. emergency care Visits 1.58 2.72 1.59 2.78 1.58 2.7 1.58 2.82 1.52 2.41 1.68 2.57
Primary care visits since last pres. 4.67 7.71 4.47 7.71 4.66 8.12 4.58 7.74 4.6 7.45 4.94 8.22
Hospital visits since last pres. 0.0909 0.338 0.0874 0.318 0.0844 0.328 0.0877 0.331 0.0889 0.324 0.0982 0.353
Emergency visits since last pres. 0.128 0.488 0.123 0.43 0.122 0.435 0.126 0.481 0.125 0.444 0.131 0.456
PDC 0.786 0.41 0.786 0.41 0.786 0.41 0.785 0.411 0.782 0.413 0.791 0.407

Splitting strategy Temporal patient Temporal recent

Patient age at prescription 66.9 12.4 67.3 12.3 67.4 12.4 66.9 12.3 67.5 12.1 67.6 12.6
Patient Gender∗ 0.513 0.5 0.517 0.5 0.515 0.5 0.517 0.5 0.509 0.5 0.509 0.5
Prescriber Age 41.6 11.1 42.2 10.9 42.2 10.7 41.5 11.2 42.4 10.6 42.0 10.4
Prescription quantity 1.06 0.319 1.06 0.319 1.06 0.325 1.06 0.31 1.06 0.326 1.08 0.372
Num. concurrent drugs 1.83 0.936 1.86 0.9 1.82 0.882 1.81 0.905 1.85 0.914 1.83 0.952
Num. prescriptions in year 1.0 1.5 1.07 1.45 1.01 1.43 0.997 1.46 1.0 1.5 1.1 1.55
Num. different drug classes 1.54 1.53 2.3 1.31 2.32 1.33 1.57 1.44 2.08 1.49 2.29 1.65
Years since index date of drug 1.47 0.756 2.23 0.958 2.43 1.02 1.48 0.696 2.16 1.03 2.39 1.18
Oversupply 41.0 96.5 66.5 133.0 68.0 128.0 41.9 89.0 62.3 127.0 71.1 162.0
Num. primary care visits 38.1 50.1 49.2 56.8 52.7 59.8 38.8 48.5 50.9 59.8 53.8 65.3
Num. hospital Visits 0.973 1.75 1.1 1.99 1.15 2.05 1.0 1.77 1.08 1.95 1.16 2.07
Num. emergency care visits 1.46 2.53 1.7 3.0 1.82 3.26 1.49 2.5 1.73 2.95 1.88 3.39
Primary care visits since last pres. 4.02 7.35 6.2 8.83 5.92 8.9 4.5 7.57 5.59 8.39 4.34 7.99
Hospital visits since last pres. 0.0896 0.327 0.0942 0.351 0.0856 0.351 0.0963 0.343 0.0868 0.341 0.0599 0.275
Emergency visits since last pres. 0.117 0.462 0.147 0.484 0.155 0.504 0.126 0.474 0.142 0.483 0.113 0.451
PDC 0.757 0.429 0.826 0.379 0.819 0.385 0.769 0.422 0.819 0.385 0.83 0.376

∗
Ratio of male to female patients.

3. Results
3.1. Descriptive statistics
The training, validation and test sets include 53302, 11410, 11410
unique prescriptions of 59085 prescribing events4 from 18581 patients
(50,6% Male, 49,4% Female), for all splitting strategies. The mean and
standard deviation of predictors and outcome variable between sets are
similar for random and patients splits. Both temporal patient and
temporal recent splits show expected differences in means of predictors
expected to increase with time, such as the average number of visits,
age at prescription, years since first prescription or oversupply available
at the start of a new prescription. Table 3 shows mean and standard
deviation of all included predictors and PDC for each splitting scenario.
3.2. Refill adherence predictions
Table 4 shows the predictive performances of each optimized model
for all splitting scenarios. In all scenarios, The Random Forest(RF) and
Gradient Boosting Trees(GB) algorithm perform the best on both vali-
dation and test set using the baseline predictors and including history.
The models do not seem to overfit on the validation set and
4
Patients can have multi prescriptions per prescribing event, for example
three prescriptions on one day constitute one prescribing event.
performance on the validation set and test set is comparable. Predictive
performance was highest on the temporal split scenario excluding pa-
tients with single prescriptions from the test set with the best model
(RF) having AUCs of approximately 0.90 and 0.91 using the baseline
predictors and baseline + history, respectively. In comparison, the
lowest performance is observed for the temporal split having the most
recent prescription in the test set with AUCs of the best model (GB) of
approximately 0.77 and 0.80 with baseline predictors and base-
line + history, respectively.
In general, including historical information by adding features of
past prescriptions (including PDC) increases prediction performance of
future adherence.
3.2.1. Predictive performance on index prescriptions
For the random, patient and temporal recent split, we evaluate the
models’ performances on all patients in the test sets that only received
one prescription, i.e., new patients and patients that discontinued
treatment after the first prescription. Given that such patients are new
to treatment, we expect higher variability in medication refill rates at
treatment initiation and, therefore, lower model performance on pre-
dicting adherence. Table 5 shows the predictive performance for pa-
tients with a single prescription only. We observe a significant decrease
in model performance compared to previous experiments, for all
models and data splits, indicating low informativeness of used pre-
dictors, mostly due to missing previous adherence and oversupply in-
formation.

7
A. Galozy and S. Nowaczyk Journal of Biomedical Informatics: X 6–7 (2020) 100075

Table 4
Predictive performance under investigated prediction scenarios. Bold figures indicate the highest model performance (AUC) per scenario. RF,Random Forest, LR,
Logistic Regression, GB, Gradient Boosting Trees, KNN, k-nearest neighbor.
Data Set Validation Test

Scenario Baseline History Baseline History

Random AUC Precision Recall AUC Precision Recall AUC Precision Recall AUC Precision Recall

RF 0.788 0.833 0.952 0.801 0.840 0.947 0.796 0.838 0.954 0.803 0.846 0.947
LR 0.751 0.931 0.528 0.783 0.919 0.637 0.755 0.934 0.534 0.786 0.925 0.630
GB 0.790 0.828 0.966 0.806 0.835 0.959 0.793 0.826 0.966 0.807 0.835 0.956
KNN 0.747 0.799 0.972 0.717 0.799 0.978 0.747 0.805 0.977 0.723 0.802 0.978

Patient

RF 0.766 0.922 0.588 0.779 0.898 0.701 0.772 0.934 0.602 0.790 0.906 0.715
LR 0.747 0.927 0.509 0.773 0.913 0.620 0.754 0.935 0.528 0.784 0.926 0.625
GB 0.766 0.807 0.975 0.788 0.815 0.969 0.779 0.812 0.980 0.803 0.823 0.971
KNN 0.730 0.799 0.966 0.699 0.795 0.972 0.736 0.802 0.971 0.702 0.800 0.974

Temporal patient

RF 0.878 0.894 0.985 0.896 0.899 0.987 0.899 0.898 0.988 0.914 0.899 0.987
LR 0.770 0.919 0.711 0.818 0.908 0.858 0.769 0.916 0.706 0.817 0.906 0.842
GB 0.857 0.899 0.971 0.883 0.903 0.979 0.876 0.901 0.973 0.897 0.907 0.979
KNN 0.775 0.845 0.972 0.741 0.834 0.991 0.786 0.844 0.971 0.754 0.829 0.987

Temporal recent

RF 0.786 0.924 0.707 0.808 0.898 0.830 0.768 0.932 0.664 0.783 0.903 0.810
LR 0.755 0.914 0.685 0.800 0.919 0.750 0.757 0.920 0.650 0.788 0.924 0.713
GB 0.787 0.835 0.982 0.817 0.854 0.975 0.773 0.847 0.979 0.801 0.862 0.966
KNN 0.746 0.830 0.979 0.704 0.828 0.984 0.731 0.840 0.966 0.694 0.835 0.974

Table 5 Table 6
Predictive performance of trained models on patients with a single prescribing Recall of trained models on patients with a sudden drop off in PDC on their
event. The history column refers to the models trained using baseline predictors recent prescription.
of the five most recent prescriptions. Bold figures indicate the highest model
Predictors Baseline History
performance (AUC).
Predictors Baseline History Temporal patient

Random AUC Precision Recall AUC Precision Recall RF 0.176 0.265

LR 0.285 0.586
RF 0.626 0.639 0.878 0.632 0.642 0.885 GB 0.204 0.285
LR 0.613 0.920 0.150 0.608 0.900 0.147 KNN 0.0178 0.102
GB 0.655 0.623 0.944 0.664 0.608 0.957 Temporal recent
KNN 0.639 0.614 0.947 0.629 0.610 0.950

Patient RF 0.193 0.630

LR 0.445 0.603
RF 0.643 0.891 0.257 0.633 0.891 0.257 GB 0.104 0.117
LR 0.618 0.891 0.206 0.616 0.900 0.197 KNN 0.00804 0.0643
GB 0.661 0.627 0.962 0.654 0.627 0.974
KNN 0.617 0.630 0.940 0.622 0.633 0.931

Temporal recent
3.3. Predictor influence on model output
RF 0.562 0.880 0.238 0.560 0.882 0.234
LR 0.630 0.903 0.194 0.590 0.888 0.186
We use the SHapley Additive exPlanations (SHAP)5 framework to
GB 0.581 0.767 0.967 0.586 0.764 0.976
KNN 0.550 0.763 0.905 0.548 0.761 0.890 generate a summary of local model explanations on the test set pre-
dictions to investigate feature importance [24,25]. In essence, shapely
values are computed by introducing each feature in the model step by
3.2.2. Sudden refill adherence drop-off step, attributing the change in prediction to the feature introduced and
For temporal patient and temporal recent splits, we investigate the averaging over all feature orderings [23]. The resulting explanations for
accuracy of the models on patients that refilled their recent prescription the overall highest performing models (GB) is shown in Fig. 4.
insufficiently given past high refill adherence. Predicting a sudden drop The number of available supply at the beginning of the prescription
in PDC would allow care providers to intervene early. had the most substantial influence on model output with higher values
Table 6 shows the predictive performance of the trained models on of available supply, increasing the likelihood of positive (adherent)
the test set for patients that had a sudden drop in PDC after exhibiting output. Other important features are past adherence to medication with
high refill adherence in previous prescriptions. All models tend to higher values increasing positive model output, while lower previous
predict continuing good PDC, resulting in low model performance with PDC does not necessarily result in negative model output shown by
the LR model showing the highest Recall if history is not included.
Adding history lifts Recall of all models.
5
Github: https://github.com/slundberg/shap.

8
A. Galozy and S. Nowaczyk Journal of Biomedical Informatics: X 6–7 (2020) 100075

Fig. 4. SHAP values of the test set for all splitting scenarios using the GB models. Positive values correspond to higher portability outputs of the model.
(t 1), (t 2) , as so forth indicate predictors from previous prescriptions.

significant overlap between positive and negative labels for low pre-
vious adherence.

3.4. PDC-pattern clustering & simulation

After clustering of PDC-patterns, we merge the 100 clusters into 21

two-year patterns of similar shape by inspection. “Adherence/ad-
herence” over the first two year period is the most common pattern
observed with 29,9% of instances. Patients with this pattern regularly
refill medication without any major gap (>7 days) in estimated supply.
The second most frequent pattern we observe is “Late first pickup/ad-
herence” (19.1%). Patients delay the first fill of medication. After the
first pickup, they regularly refill without major gaps. The third most
common patterns are “Single gap/adherence” (16%), i.e., patients fill
their first prescription on time but do not refill for an extended amount
of time (>7 days), and refill regularly afterwards. Less common patterns
are: “Adherence/early discontinue” (8%), “Adherence/single gap”
(5.8%), “Early discontinue/nonadherence”6 (5.17%), “Adherence/late
first pickup” (3.81%), “Adherence/ nonadherence” (2.82%) and Fig. 5. Six most common two-year PDC-patterns of patients with index pre-
“Nonadherence/nonadherence” (2.19%). We observe distinct patterns scription. Four out of six patterns show continuous refilling after the first year,
of “Half dosing/Half dosing” (1.59%). i.e., patients only refill their with two patterns showing discontinuation after the first refill or first year.
mediation half as often. There is no indication in the prescription record
that points towards halving the dosage for those patients from the patterns is shown in Fig. 6. The “Adherence/Adherence” pattern is the
prescriber’s perspective. Fig. 5 shows the six most common PDC-pat- simplest to model given that the model parameters can be chosen in
terns over two years. such a way that motivation does not decrease with time. Simulated
patients that exhibit such patterns are necessarily consuming their
3.4.1. Simulation medication at levels that would lead to regular refilling, i.e., supply is
The best fit simulation patterns for the six most common PDC- exhausted. All models can reproduce the pattern early discontinue/
nonadherence, being mostly independent on the simulated consump-
tion pattern, provided that the initial supply is consumed sporadically
6
“nonadherence” here refers to total lack of fill not to be confused with the over the two years. Other PDC-patterns are not as easily reproduced
definition of nonadherence in the prediction study, i.e. PDC <80% .

9
A. Galozy and S. Nowaczyk
Fig. 6. Best fit simulated medication-taking patterns for the top six PDC-patterns for each simulation model. The overall best fit is achieved using the sigmoidal model
of medication-taking, with early “discontinue/nonadherence” and “adherence/adherence” patterns being easily reproduced by most models.
with the proposed models. Given that pickup probability is directly tied
to available supply, a “late pickup/adherence” pattern can only be
explained in two ways: Either the patient has medication available at
the beginning of the prescription, or low motivation to fill their first
prescription. An “Adherence/early discontinue” pattern without gaps in
the first year is reproduced my models exhibiting consistent medica-
tion-taking with a drop in motivation shortly after the last refill. In
constant, seasonal and exponential models cannot reproduce such
patterns given the simulated medication-taking behaviour.
The sigmoidal model most faithfully reproduces the “Adherence/
single gap” pattern. After consistent medication-taking behaviour, we
see a motivation drop-off close the supply exhaustion after the first refill
in the second year with eventual re-uptake of treatment.
4. Discussion
Overall model performance for medication adherence prediction is
surprisingly high, with AUCs between 0.78–0.91 on the test set for all
investigated splits. The available oversupply has the highest impact on
model decisions, with higher oversupply values correlating with in-
creased PDC. This fact is not surprising, given that, firstly, oversupply
10
Journal of Biomedical Informatics: X 6–7 (2020) 100075
does contribute to PDC directly and, secondly, patients that are regular
refillers (adherent patients), do tend to accumulate a significant supply
as they are progressing in their treatment, due to extra supply dispensed
(100 days supply every 90 days).
The highest performance on the test set is achieved in the temporal
splitting strategy, where new patients are excluded. It has to be noted
that predicting adherence for a self-selected group of patients, that are
already several years into their treatment, are expected to be somewhat
adherent to treatment. This selection bias is also shown when pre-
dicting adherence using the “temporal recent strategy” seeing a drop in
AUC to 0.78 when new patients are introduced that do not have a prior
treatment history.
Predictive performance is modest for patients with a single pre-
scription for all splitting scenarios with AUCs between 0.56 and 0.65,
illustrating again that clinical predictors are only weakly correlated
with future medication adherence. Most recent work that investigated
the predictive performance of clinical predictors come to similar con-
clusions, showing that initial medication filling behaviour predicts refill
adherence more accurately for patients with index prescription [13]. A
similar low performance is observed for patients with a sudden drop-off
in PDC, where previous PDC has been high, with model Recall ranging
A. Galozy and S. Nowaczyk
from under 1% (KNN) to 44%, not including history and 6–63% when
adding historical information. The models are unable to perform well
on patients where previous information is missing or predict adherence
for prescription with a sudden drop in PDC for otherwise adherent
patients. This model behaviour shows a clear bias towards patients that
remain in treatment with regular follow-ups, i.e., patients that remain
refill adherent also continue to utilize healthcare resources, generating
a data bias towards adherence. This observation might partially explain
why adding historical information from past prescriptions, and ad-
herence improves prediction performance on the test set for all testing
scenarios. The fact that patients had regular checkups in the past pro-
vides increased confidence that they remain refill adherent, as evi-
denced by past adherence being among the strongest predictors of fu-
ture refill adherence.
Given these results and results of previous studies, it is unlikely that
EHRs, in their current state, provide information relevant for refill
adherence that allows the development of high-performance models
that can predict refill adherence at the time when the prescription is
issued. It might be more useful to switch the focus from prediction to
monitoring, analyzing refill patterns and provide patients with ade-
quate interventions.
All of the common PDC-patterns can be reproduced by the simula-
tion models investigated, coupling mediation refilling with medication-
taking behaviour. Early discontinuation in the first year exhibits the
most diverse of all consumption patterns. Any consumption model that
does not exhaust the available supply can reproduce such patterns,
pointing towards high variability in possible consumption patterns with
significant differences in levels of drug exposure. Good or near-perfect
refill adherence, given our simulation models, is only possible if con-
sumption patterns are consistent as well, i.e., to avoid significant gaps
covered days, patients are not permitted to have significant gaps in
medication-taking.
Other patterns are more difficult to model, restricting the set of
possible medication consumption patterns. In all cases, the sigmoidal
model does fit the patterns the best, reproducing the PDC-patterns
somewhat faithfully, indicating medication consumption that exhibits
periods of nonadherence instead of constant are regular forgetting of
medication. The exponential and constant models do not reproduce the
most common PDC-patterns beyond early discontinuation. These
models seem to provide the least plausible consumption patterns for the
majority of cases.
Even though the real pattern of mediation use is not uniquely re-
trievable from dispensation information, the realm of possibilities is
somewhat restricted given a link between refilling and actual mediation
use does exist when examining the phenomenon of adherence on po-
pulation level.
5. Conclusion
We investigated the predictability of refill mediation adherence
under various data splitting strategies using EHR and dispensation re-
cords. Model performance is lowest under a realistic model deployment
scenario, with very low discriminability on patients with index pre-
scriptions or when a sudden drop in adherence occurs. This indicates
data bias due to overrepresentation of chronic disease patients with
high healthcare utilization patterns associated with high refill ad-
herence. When such bias is not taken into consideration, it is unlikely
that prediction models developed using EHR, in their current state, are
particularly useful for targeting specific patients for interventions at the
time of prescription. Nonetheless, we have shown that introducing
historical information does improve prediction recall of sudden changes
in adherence, pointing towards the importance of analyzing adherence
longitudinally and training models using information from past
11
Journal of Biomedical Informatics: X 6–7 (2020) 100075
behaviour. Predicting refill adherence on a prescription basis remains
challenging when initial filling behaviour is not considered, pointing
towards the necessity to monitor patients adherence continuously and
providing interventions just in time when patterns of nonadherence
start manifesting.
Our simulations show that particular patterns of medication con-
sumption are incompatible with observed PDC-patterns of patients with
index prescription. Provided there exists a link between refill and real
adherence, patterns of medication-taking might not be as arbitrary, and
knowing how patients refill their medication can provide insights into
possible medication-taking habits. Such habits could then be analyzed
in terms of medication effectiveness, their effects on clinical outcomes
and finding appropriate interventions.
6. Limitations
Our study has several limitations. We analyzed data from patients
that are located in a specific region in Sweden with healthcare and
pharmacy policies that might not reflect systems in other countries. We
expect refill patterns to change accordingly.
We do not investigate the frequency and timeliness of dispensations,
but show the number of covered days given the dispensed supply and
prescribed daily dose. As such, simulations do not reflect dispensation
patterns per se, but dispensations that would lead to similar PDC-pat-
terns. Furthermore, our simulation models assume patients either forget
to take their mediation or consume their daily dose as prescribed. We
did not simulate over-consumption of medication, i.e., patients might
exhaust their supply before the next official refill.
For the refill pattern analysis, we mainly focused on patients with
monotherapy. Although we include patients with combination drugs,
patients treated for hypertension are often prescribed multiple medi-
cations and may exhibit different patterns not investigated in this study.
We do not validate our simulation results with actual consumption
patterns of the cohort under investigation. Given that such data is not
available, we focused on simple patterns, but real consumption beha-
viours might be more complex.
Authors’ statement
All authors certify that they have seen and approved the final ver-
sion of the manuscript. The article is the authors’ original work, was not
published prior or is under consideration for publication elsewhere. All
authors have made a substantial contribution to this study. Galozy:
Study concept and design, data analysis, data interpretation and
drafting of the manuscript. Nowaczyk: Study concept and design, data
interpretation and critical revision of the manuscript for important in-
tellectual content.
Declaration of Competing Interest
The authors declare that they have no known competing financial
interests or personal relationships that could have appeared to influ-
ence the work reported in this paper.
Acknowledgements
We thank PhD, Mattias Ohlsson for his support during research.
Funding: The authors thank Vinnova, Health Technology Center and
CAISR at Halmstad University and Hallands Hospital for financing the
research work under the project iMedA [Grant No.: 2017-04617]. We
thank Group Inc. Consultants to Government and Industries (CGI)
Sweden and Region Halland for granting data and legal access.
A. Galozy and S. Nowaczyk Journal of Biomedical Informatics: X 6–7 (2020) 100075

Appendix A. Hyper-parameters grid-search for prediction models

All prediction models underwent hyper-parameter grid-search on the validation set to find the optimal combination of parameters yielding the
best AUC. The parameters considered for optimization of each algorithm are shown in Table A.1. The optimal parameter combinations for each
algorithm and splitting strategy is shown in Table A.2. For all algorithm implementations, training and evaluation we used the scikit-learn package
implemented in the programming language python [28].

Table A.1
Grid-search hyper-parameters for each algorithm.
Algorithm Hyper-parameters

RF n estimators: 100, 150, 200, 250, 300, 400, 500

max depth: 3, 10, 12, 15, None
criterion: entropy, gini

LR C: 0.1, 0.5, 0.8, 1.0, 2.0, 5.0

penalty: l1, l2

GB n estimators: 100, 150, 200, 250, 300, 400, 500

max depth: 3, 10, 12, 15, None

KNN n neighbors: 1, 3, 5, 10, 15, 20, 25

p: 1, 2, 3

Table A.2
Optimal parameters for each algorithm, predictor set and splitting strategy.
Baseline History

Algorithm RF LR GB KNN RF LR GB KNN

Splitting strategy

Random n estim.: 500 C:0.8 n estim.: 100 n neighbors: 25 n estim.: 400 C:0.1 n estim.: 100 n neighbors: 25
max depth: None penalty: l2 max depth: 15 p: 1 max depth: None penalty: l1 max depth: 15 p: 1
criterion: entropy criterion: entropy

Patient n estim.: 500 C:0.1 n estim.: 300 n neighbors: 25 n estim.: 300 C:0.1 n estim.: 150 n neighbors: 25
max depth: 10 penalty: l2 max depth: 3 p: 1 max depth: 10 penalty: l1 max depth: 3 p: 3
criterion: entropy criterion: entropy

Temporal patient n estim.: 500 C:0.1 n estim.: 500 n neighbors: 20 n estim.: 500 C:0.1 n estim.: 250 n neighbors: 25
max depth: None penalty: l1 max depth: 15 p: 1 max depth: None penalty: l1 max depth: 15 p: 1
criterion: gini criterion: entropy

Temporal recent n estim.: 400 C:0.1 n estim.: 400 n neighbors: 25 n estim.: 500 C:0.1 n estim.: 200 n neighbors: 25
max depth: 12 penalty: l1 max depth: 3 p: 1 max depth: 10 penalty: l1 max depth: 3 p: 1
criterion: entropy criterion: entropy

References
[1] T.M. Abegaz, Y.G. Tefera, B.A. T., Target organ damage and the long term effect of
nonadherence to clinical practice guidelines in patients with hypertension: a ret-
rospective cohort study, Int. J. Hypertens. 2090-0384 (2017) 749, https://doi.org/
10.1155/2017/2637051.
[2] A. Galozy, S. Nowaczyk, A. Sant’Anna, M. Ohlsson, M. Lingman, Pitfalls of medi-
cation adherence approximation through ehr and pharmacy records: Definitions,
data and computation, Int. J. Med. Informatics 136 (2020) 104092, https://doi.org/
10.1016/j.ijmedinf.2020.104092 http://www.sciencedirect.com/science/article/
pii/S1386505619308974.
[3] S.E. Andrade, K.H. Kahler, F. Frech, K.A. Chan, Methods for evaluation of medi-
cation adherence and persistence using automated databases, Pharmacoepidemiol.
Drug Saf. 15 (2006) 565–574, https://doi.org/10.1002/pds.1230.
[4] A. Ashfaq, S. Lönn, H. Nilsson, J.A. Eriksson, J. Kwatra, Z.M. Yasin, J.E. Slutzman,
T. Wallenfeldt, Z. Obermeyer, P.D. Anderson, M. Lingman, Data resource profile:
regional healthcare information platform in Halland, Sweden, a dedicated en-
vironment for healthcare research, Int. J. Epidemiol. (2020), https://doi.org/10.
1093/ije/dyz262.
[5] P.C. Baumgartner, R.B. Haynes, K.E. Hersberger, I. Arnet, A systematic review of
medication adherence thresholds dependent of clinical outcomes, Front. Pharmacol.
9 (2018), https://doi.org/10.3389/fphar.2018.01540.
[6] H.B. Bosworth, B.B. Granger, P. Mendys, R. Brindis, R. Burkholder,
S.M. Czajkowski, J.G. Daniel, I. Ekman, M. Ho, M. Johnson, S.E. Kimmel, L.Z. Liu,
J. Musaus, W.H. Shrank, E.W. Buono, K. Weiss, C.B. Granger, Medication ad-
herence: a call for action, Am. Heart J. 162 (2011) 412–424, https://doi.org/10.
1016/j.ahj.2011.06.007.
[7] M. Burnier, Is there a threshold for medication adherence? Lessons learnt from
electronic monitoring of drug adherence, Front. Pharmacol. 9 (2019), https://doi.
org/10.3389/fphar.2018.01540.
[8] M. Burnier, B.M. Egan, Adherence in hypertension, Circ. Res. 124 (2019)
1124–1140, https://doi.org/10.1161/CIRCRESAHA.118.313220 https://www.
ahajournals.org/doi/abs/10.1161/CIRCRESAHA.118.313220.
[9] S.M. Cadarette, L. Wong, An introduction to health care administrative data, Can. J.
Hospital Pharmacy 68 (2015) 232–237, https://doi.org/10.4212/cjhp.v68i3.1457.
[10] R.L. Cutler, F. Fernandez-Llimos, M. Frommer, C. Benrimoj, V. Garcia-Cardenas,
Economic impact of medication non-adherence by disease groups: a systematic
review, BMJ Open 8 (2018) e016982, https://doi.org/10.1136/bmjopen-2017-
016982.
[11] Ester, M., peter Kriegel, H., Sander, J., Xu, X., 1996. A density-based algorithm for
discovering clusters in large spatial databases with noise, AAAI Press. pp. 226–231.
doi:10.5555/3001460.3001507. URL https://dl.acm.org/doi/10.5555/3001460.
3001507.
[12] J.M. Franklin, A.A. Krumme, W.H. Shrank, O.S. Matlin, T.A. Brennan,
N.K. Choudhry, Predicting adherence trajectory using initial patterns of medication
filling, Am. J. Manag. Care 21 (2015) e537–544.
[13] J.M. Franklin, W.H. Shrank, J. Lii, A.K. Krumme, O.S. Matlin, T.A. Brennan,

12
A. Galozy and S. Nowaczyk
N.K. Choudhry, Observing versus predicting: initial patterns of filling predict long-
term adherence more accurately than high-dimensional modeling techniques,
Health Serv. Res. 51 (2016) 220–239.
[14] R. Grymonpre, M. Cheang, M. Fraser, C. Metge, D.S. Sitar, Validity of a prescription
claims database to estimate medication adherence in older persons, Med. Care 44
(2006).
[15] R.L. Harrison, Introduction to Monte Carlo simulation, AIP Conf. Proc. 1204 (2010)
17–21.
[16] F.D.R. Hobbs, Cardiovascular disease: different strategies for primary and sec-
ondary prevention? Heart 90 (2004) 1217–1223. doi:10.1136/hrt.2003.027680,
arXiv:https://heart.bmj.com/content/90/10/1217.full.pdf. https://heart.bmj.com/
content/90/10/1217.
[17] V. Kini, P.M. Ho, Interventions to improve medication adherence: a review, JAMA
320 (2018) 2461–2473, https://doi.org/10.1001/jama.2018.19271.
[18] A.A. Krumme, J.M. Franklin, D.L. Isaman, O.S. Matlin, A.Y. Tong, C.M. Spettell,
T.A. Brennan, W.H. Shrank, N.K. Choudhry, Predicting 1-year statin adherence
among prevalent users: a retrospective cohort study, J. Manag. Care Specialty
Pharmacy 23 (2017) 494–502, https://doi.org/10.18553/jmcp.2017.23.4.494.
[19] A.A. Krumme, G. Sanfélix-Gimeno, J.M. Franklin, D.L. Isaman, M. Mahesri,
O.S. Matlin, W.H. Shrank, T.A. Brennan, G. Brill, N.K. Choudhry, Can purchasing
information be used to predict adherence to cardiovascular medications? An ana-
lysis of linked retail pharmacy and insurance claims data, BMJ Open 6 (2016)
e011015, https://doi.org/10.1136/bmjopen-2015-011015 http://bmjopen.bmj.
com/content/6/11/e011015.abstract.
[20] H. Kumamaru, M.P. Lee, N.K. Choudhry, Y.H. Dong, A.A. Krumme, N. Khan,
G. Brill, S. Kohsaka, H. Miyata, S. Schneeweiss, J.J. Gagne, Using previous medi-
cation adherence to predict future adherence, J. Managed Care Specialty Pharmacy
24 (2018) 1146–1155, https://doi.org/10.18553/jmcp.2018.24.11.1146.
[21] W.Y. Lam, Paula Fresco, Medication adherence measures: an overview, BioMed Res.
Int. 1–12 (2015), https://doi.org/10.1155/2015/217047.
[22] J.C. Lauffenburger, J.M. Franklin, A.A. Krumme, W.H. Shrank, O.S. Matlin,
C.M. Spettell, G. Brill, N.K. Choudhry, Predicting adherence to chronic disease
13
Journal of Biomedical Informatics: X 6–7 (2020) 100075
medications in patients with long-term initial medication fills using indicators of
clinical events and health behaviors, J. Managed Care Specialty Pharmacy 24
(2018) 469–477, https://doi.org/10.18553/jmcp.2018.24.5.469 pMID: 29694288.
https://doi.org/10.18553/jmcp.2018.24.5.469.
[23] S.M. Lundberg, G. Erion, H. Chen, A. DeGrave, J.M. Prutkin, B. Nair, R. Katz,
J. Himmelfarb, N. Bansal, S.I. Lee, From local explanations to global understanding
with explainable ai for trees, Nat. Mach. Intell. 2 (2020) 2522–5839.
[24] S.M. Lundberg, S.I. Lee, A unified approach to interpreting model predictions, in:
I. Guyon, U.V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan,
R. Garnett (Eds.), Advances in Neural Information Processing Systems 30, Curran
Associates, Inc., 2017, pp. 4765–4774.
[25] S.M. Lundberg, B. Nair, M.S. Vavilala, M. Horibe, M.J. Eisses, T. Adams, D.E. Liston,
D.K.W. Low, S.F. Newman, J. Kim, et al., Explainable machine-learning predictions
for the prevention of hypoxaemia during surgery, Nat. Biomed. Eng. 2 (2018) 749.
[26] J. MacQueen, Some methods for classification and analysis of multivariate ob-
servations, in: Proceedings of the Fifth Berkeley Symposium on Mathematical
Statistics and Probability, Statistics, vol. 1, University of California Press, Berkeley,
Calif, 1967, pp. 281–297. https://projecteuclid.org/euclid.bsmsp/1200512992.
[27] J. Paparrizos, L. Gravano, k-shape: Efficient and accurate clustering of time series,
ACM SIGMOD Record 45 (2016) 69–76, https://doi.org/10.1145/2949741.
2949758.
[28] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel,
M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos,
D. Cournapeau, M. Brucher, M. Perrot, E. Duchesnay, Scikit-learn: machine learning
in Python, J. Mach. Learn. Res. 12 (2011) 2825–2830.
[29] D. Prakash, Target organ damage in newly detected hypertensive patients, J. Family
Med. Prim. Care 8 (2019) 2042–2046.
[30] J.F. Steiner, P.M. Ho, B.L. Beaty, L.M. Dickinson, R. Hanratty, C. Zeng, H.M. Tavel,
E.P. Havranek, A.J. Davidson, D.J. Magid, R.O. Estacio, Sociodemographic and
clinical characteristics are not clinically useful predictors of refill adherence in
patients with hypertension, Circ. Cardiovasc. Qual. Outcomes 2 (2009) 451–457,
https://doi.org/10.1161/CIRCOUTCOMES.108.841635.