Professional Documents
Culture Documents
fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JBHI.2016.2614991, IEEE Journal of
Biomedical and Health Informatics
JBHI-00268-2016.R1 1
2168-2194 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JBHI.2016.2614991, IEEE Journal of
Biomedical and Health Informatics
JBHI-00268-2016.R1 2
2168-2194 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JBHI.2016.2614991, IEEE Journal of
Biomedical and Health Informatics
JBHI-00268-2016.R1 3
2168-2194 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JBHI.2016.2614991, IEEE Journal of
Biomedical and Health Informatics
JBHI-00268-2016.R1 4
TABLE I
are randomly selected when building a single tree. A 10-fold STATISTICAL CHARACTERISTICS OF THE FINAL MULTI-LABEL DATASET
cross validation was performed to test the accuracy of the Dataset statistics measures Count (n%)
multi-label recommendation model. To validate the Total 2542
performance of the ensemble algorithm, two other popular Number of patient features 77
multi-label classifiers were compared.
Label cardinality 1.552
Multi-Label k-Nearest Neighbor (ML-kNN): k-nearest
neighbor techniques were adapted to address multi-label data in Input features Count (n%) Mean (±SD)
which a maximum a posteriori (MAP) rule was utilized to make Gender
a prediction by reasoning with the labeling information
Male 1668 (0.656)
embodied in the neighbors, proposed by Zhang et al [31]. The
number of neighbors, k, was set to 10, and the smoothing factor Female 874 (0.344)
was set to 1. Age 66.46 (±13.81)
• Random k-Labelsets (RAkeL): a small random subset of (18, 45] 173 (0.068)
labels and learning of a single-label classifier was considered (45, 70] 1280 (0.504)
for the prediction of each element in the powerset of this subset >70 1089 (0.428)
[32]. Random Forest was used as the base binary learner. Concomitant
The performance measures for multi-label classification diseases (CD)
problems can be categorized into two groups: example-based With no CD 927 (0.365)
metrics and label-based metrics [26]. In this study, we With one CD 1127 (0443)
employed four example-based measures: Hamming Loss,
With 2 CDs 362 (0.142)
𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦𝑒𝑥𝑎𝑚 , 𝑅𝑒𝑐𝑎𝑙𝑙𝑒𝑥𝑎𝑚 , and 𝐹𝑒𝑥𝑎𝑚 .
With 3 or more
Hamming Loss evaluates the fraction of misclassified 126 (0.050)
CDs
instance-label pairs, in which is a relevant label is missed or an Vital sign
irrelevant label is predicted; defined as Equation (1), in which h values
denotes the multi-label classifier; 𝑝 is the number of test sets; 𝑌𝑖 Systolic blood
132.58 (±17.71)
is the true set of test data; ℎ(𝑥𝑖 ) is the predicted set of test data; pressure (mmHg)
and ∆ stands for the symmetric difference between two sets. Diastolic blood
1 𝑝 76.78 (±11.11)
pressure (mmHg)
Hloss(h) = ∑𝑖=1 |ℎ(𝑥𝑖 )∆𝑌𝑖 | (1)
𝑝
And 𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦𝑒𝑥𝑎𝑚 , 𝑅𝑒𝑐𝑎𝑙𝑙𝑒𝑥𝑎𝑚 , and 𝐹𝑒𝑥𝑎𝑚 are defined as Temperature (℃) 36.26 (±2.21)
Equation (2)-(4).
1 𝑝 |𝑌 ⋂ℎ(𝑥𝑖 )| Pulse rate (/min) 75.30 (±9.52)
𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦𝑒𝑥𝑎𝑚 (h) = ∑𝑖=1 𝑖 (2)
𝑝 |𝑌𝑖 ⋃ℎ(𝑥𝑖 )|
1 𝑝 |𝑌𝑖 ⋂ℎ(𝑥𝑖 )| Output labels Count
𝑅𝑒𝑐𝑎𝑙𝑙𝑒𝑥𝑎𝑚 (h) = ∑ (3)
𝑝 𝑖=1 |𝑌 | 𝑖
2×|𝑌𝑖 ⋂ℎ(𝑥𝑖 )| Alpha-glucosidase inhibitors 953
𝐹𝑒𝑥𝑎𝑚 (h) = |𝑌𝑖 |+|ℎ(𝑥𝑖 )|
(4)
DPP-4 inhibitors 24
Meglitinides 163
III. RESULTS GLP-1 receptor inhibitors 11
A. Recommendation model performance and validation Insulin 1525
After data cleaning, 2,542 cases were included with eight Metformin 788
medication labels and 77 patient attributes that contained two Sulfonylureas 415
demographic variables, 11 concomitant diseases, five vital sign Thiazolidinediones 66
values, and 57 lab test results. The gender and concomitant
diseases variables were binary, the age and vital sign value antihyperglycemic medications were prescribed to each patient
variables were continuous and regularized, and the lab test on average, was defined as Equation (5), where 𝐷 is the number
results variables were nominal because the structured lab test of cases and 𝑌𝑖 represents the label of the 𝑖 case.
1 |𝐷|
results included three statuses in the hospital EHR: H (test LC(D) = ∑𝑖=1 |𝑌𝑖 | (5)
|𝐷|
results were higher than the upper limit of the normal range); N
As shown in Table I, a class-imbalance phenomenon was
(test results fell in the normal range); and L (test results were
evident after decomposition of the multi-label dataset into eight
lower than the lower limit of the normal range). Table I lists the
single-label datasets. Seventy percent of the 2542 cases were
statistical characteristics of the patients in the final dataset. In
included in the training set, and the others were included in the
total, 65.6% of the patients were male, and 93.2% of the
test set. Fig. 4 illustrates the average performances of 10-fold
patients were older than 45 years old. Of the eight output
cross validation single-label Random Forest classifiers for each
medication classes, insulin was prescribed to the majority of
antihyperglycemic medication label with or without SMOTE.
patients whereas GLP-1 receptor inhibitors were prescribed to
If SMOTE was not applied before Random Forest, the classifier
the least number of patients (only 11). The Label Cardinality
accuracy was high in medication labels where class-imbalance
(LC) value was 1.552, which means that 1.552 classes of
was severe, such as DPP-4 inhibitors (0.9906) and GLP-1
2168-2194 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JBHI.2016.2614991, IEEE Journal of
Biomedical and Health Informatics
JBHI-00268-2016.R1 5
receptor inhibitors (0.9958), and relatively low in medication other two methods.
labels where positive cases and negative cases were relatively
B. System implementation
balanced, such as α-glycosidase inhibitors (0.6603) and insulin
(0.6648). After applying SMOTE, the classifier accuracy for According to the above designed framework and SDM
each medical label was found to be stable. Recall measure process, a T2DM SDM system prototype was developed that
indicates the classifier’s ability to predict positive cases. Fewer can run on Android smart devices. EHR data were extracted
using an Ensemble platform (Intersystems, Cambridge, MA,
positive cases in the dataset resulted in a more severe
United States.), and the BSR model was built using R language.
class-imbalance and lower recall. Without SMOTE, the recall
Fig. 5 shows two screenshots of the T2DM SDM system.
During the choice talk phase, physicians can help patients
understand their specific conditions and review their clinical
history using the clinical history review panel (Fig. 5a). In the
option talk and decision talk phases, physicians use the same
interface (Fig. 5b). Both panels in Fig. 5 are mainly divided into
three parts: basic patient information (red region in Figs. 5a and
5b), current vital sign values (purple region in Figs 5a and 5b),
and the clinical history review as a timeline (Fig. 5a) or a list of
medication label. In severe class-imbalanced labels, the Fig. 5. Screenshots of the T2DM SDM system interface.
F-measure was relatively lower than that in class-balanced
labels, which was caused by false positive prediction. antihyperglycemic medications with their corresponding
To validate the overall performance of the multi-label benefits and harms (Fig. 5b, orange region).
recommendation model, the prediction results were assembled The region describing the patient’s basic information
and compared to two popular multi-label classification includes basic demographic information and current admission
algorithms: ML-kNN and RAKEL. The example-based administration information. The patient’s temperature, blood
measure of the three methods is shown in Table II. The average pressure, pulse rate, weight, HbA1c, and blood sugar values are
accuracy after 10-fold cross validations was 0.7611, the recall displayed in the patient’s current vital sign values region. In the
was 0.9664, and the F-measure was 0.8269, which was much pink region shown in Fig. 5a, the clinical history of the patient
higher than the results of the other two methods. For Hamming is displayed as a timeline. If a lab test report or an examination
Loss, the BSR was 0.0918; thus, it performed better than the report is available, the detailed report will be obtained by
clicking on the report name (Fig. 5a, blue region). Physicians
2168-2194 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JBHI.2016.2614991, IEEE Journal of
Biomedical and Health Informatics
JBHI-00268-2016.R1 6
can use the button in the blue circle shown in Fig. 5a to navigate tailored the option information by giving patients and
to option and decision talk panel (Fig. 5b). Similarly, if physicians an intuitive and clear sense of what decisions may
physicians want to help the patient review his/her clinical be appropriate based on the patient values and guiding them to
history and detailed clinical information during the option talk focus on the specific knowledge that they may need the most.
or decision talk phases, they can click the button in the blue Because glycemic control is an issue that is repeated in every
circle shown in Fig. 5b. The orange region (Fig. 5b) is the main encounter, and changes in a patient’s lifestyle and other
component of the DA tool and includes the following: names of conditions may affect the choice of mellitus medications,
the eight types of antihyperglycemic medications, HbA1c diabetes medication choice should not be established only once
control effects, daily routine, weight effects, hypoglycemia or only depend on “static” knowledge. Compared with a
risks, estimated costs per day, and other side effects. The
well-known T2DM DA, Diabetes Medication Choice [14], the
estimated costs per day are extracted by the Ensemble platform
EHR data utilization in our T2DM SDM system placed more
from the current hospital EHR price list and listed by specific
emphasis on the patient’s current condition, and this
medication, for example “acarbose: 4.92-7.41”. Additionally,
the recommended medication resulting from the BSR is decision-making process could be repeated easily and
highlighted in green. effectively.
DAs are being developed primarily in North America,
IV. DISCUSSION Europe, and Australia. Policy makers in these regions, such as
the National Health Service in the United Kingdom [35] and the
In this study, a T2DM SDM system was developed that
Institute of Medicine in the United States [4], have been making
followed IPDAS criteria and Elwyn’s SDM model to help
efforts to involve patients in decision making. Moreover,
patients and their physicians have a conversation about
Malaysia is a pioneer of SDM promotion in Asia where
antihyperglycemic medications. In addition to knowledge
physicians are aware of SDM but few DAs are used in practice
retrieved from guidelines telling patients the benefits and harms
[36]. In China, the concept of SDM remains unfamiliar to
of each mellitus medication, we utilized patient information in
clinical professionals, let alone DA implementations. Few
hospital EHRs to make a medication recommendation list to
studies have investigated SDM in Chinese patients, and none
provide decision support. We applied an ensemble multi-label
involve patients in Mainland China [37]. Due to the scarce
classification method, BSR, to the real-world EHR data to
healthcare resources in China, a physician may treat more than
achieve the reuse of EHR data and outstanding classifier
100 patients per day; therefore, the communication time is
performance, particularly in the prediction of the minority
limited. SDM with tailored knowledge is an effective method to
positive cases. Although the T2DM SDM system described in
improve the communication between physicians and patients.
this manuscript was designed to be physician-guided, patients
Therefore, the promotion of SDM in China may be an
can still play an active role in the SDM process. Through the
opportunity to enhance the trust between physicians and
clinical history review panel, patients can use the DAs to
patients and make the consultation more pertinent. However,
review their health conditions and realize that they need an
evidence medicine and HIT have been undergoing rapid in
antihyperglycemic medication. In the option talk phase, the DA
recent years, and they have laid the foundation for the
system lists all of the medication options with their usage
promotion of SDM and DA tools. To the best of our knowledge,
guidelines, benefits and harms, whereas in the decision talk
this study constitutes the first attempt for the development of a
phase, patients can talk to their physicians more effectively
DA that was designed and developed for patients in Mainland
according to the specific medications recommended by the
China. When designing the system content and user interfaces,
system and therefore aid the decision-making process. This
Chinese T2DM guidelines and real-world Chinese hospital
SDM process is interactive and would not be successful in the
EHR data were taken into account, as were the HIT utilization
absence of either the physician or patient.
customs in today’s Chinese healthcare.
The relative dependent system framework, compared to the
The T2DM SDM system and the process used in its
DAs embedded into EHRs, has the advantages of feasibility
development have several limitations, and certain work is
and maintenance. When bringing this framework into practice
needed in the future. First, the model was built only from EHR
in the clinic workflow, hospital information technology
data for hospitalized T2DM, and the outcome of the medication
engineers only need to provide certain data interfaces rather
use was not taken into account. However, it is known that
than system redevelopment. Once the model or knowledge
antihyperglycemic medication decisions are also commonly
required for decision support is changed or outdated,
made in an outpatient setting and that the T2DM medication
modification of the T2DM SDM system will not affect the
strategy decision is often based on previous outcomes of
normal operation of the hospital’s EHR system.
medication use. With the future development of HIT in China,
Idiomatically, physicians help patients tailor knowledge in
medical records for outpatient settings and clinical outcome
clinical encounters according to their conditions. However, in
could be better recorded, and the method proposed in this study
recent decades, the “Meaningful Use” Regulation for EHRs
could be easily extended. Second, the proposed
have not only promoted the adoption of EHRs worldwide but
recommendation model and clinical guidelines serve as
has also resulted in a large amount of high-quality EHR data;
decision support basis separately in the T2DM SDM system.
thus, the reuse of EHR data has become an alternative method
Physicians need to combine the knowledge of guidelines and
[33], [34]. The recommendation model applied in this study
the results of the recommendation model to guide the patients
2168-2194 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JBHI.2016.2614991, IEEE Journal of
Biomedical and Health Informatics
JBHI-00268-2016.R1 7
make decisions. Although the workflow of the DA, in which [7] C. Rathert, M. D. Wyrwich, and S. A. Boren, “Patient-Centered Care and
Outcomes: A Systematic Review of the Literature,” Medical Care
the physicians make the final decision and the crowd wisdom Research and Review, vol. 70, no. 4, pp. 351-379, August 1, 2013, 2013.
theory [38] can avoid the risk of wrong decisions to some extent, [8] M. J. Barry, and S. Edgman-Levitan, “Shared decision making—the
the inclusion of a guideline check in the recommendation pinnacle of patient-centered care,” N Engl J Med, vol. 366, no. 9, pp.
780-781, 2012.
model in the future would be an effective way to avoid the risk
[9] G. Elwyn, C. Dehlendorf, R. M. Epstein, K. Marrin, J. White, and D. L.
of wrong decisions due to wrong predictions from the Frosch, “Shared decision making and motivational interviewing:
recommendation model. Third, the current model can only be achieving patient-centered care across the spectrum of health care
evaluated using statistic methods. Effective implementation of problems,” Ann Fam Med, vol. 12, no. 3, pp. 270-275, May 1, 2014,
2014.
SDM in China demands urgent realization. To transform this [10] A. M. O'Connor, C. L. Bennett, D. Stacey, M. Barry, N. F. Col, K. B.
study from an experimental research project to a practical Eden, V. A. Entwistle, V. Fiset, M. Holmes-Rovner, S. Khangura, H.
application, further efforts are necessary to explore how the Llewellyn-Thomas, and D. Rovner, “Decision aids for people facing
health treatment or screening decisions,” Cochrane Database Syst Rev, no.
SDM implementation pattern can be adapted to Chinese 3, pp. CD001431, 2009.
conditions. In addition, in the future, an analysis of the T2DM [11] G. Elwyn, A. O'Connor, D. Stacey, R. Volk, A. Edwards, A. Coulter, R.
SDM system proposed in this manuscript after its use in Thomson, A. Barratt, M. Barry, S. Bernstein, P. Butow, A. Clarke, V.
Entwistle, D. Feldman-Stewart, M. Holmes-Rovner, H.
practical clinical settings would be valuable and promote SDM Llewellyn-Thomas, N. Moumjid, A. Mulley, C. Ruland, K. Sepucha, A.
development in China. Sykes, and T. Whelan, “Developing a quality criteria framework for
patient decision aids: online international Delphi consensus process,”
BMJ, vol. 333, no. 7565, pp. 417, 2006-08-24 22:00:11, 2006.
V. CONCLUSION [12] G. Elwyn, A. M. O'Connor, C. Bennett, R. G. Newcombe, M. Politi, M.-A.
This work focused on achieving a direct pathway between Durand, E. Drake, N. Joseph-Williams, S. Khangura, A. Saarimaki, S.
Sivell, M. Stiel, S. J. Bernstein, N. Col, A. Coulter, K. Eden, M. Härter, M.
the SDM process and patient data and proposed a SDM system H. Rovner, N. Moumjid, D. Stacey, R. Thomson, T. Whelan, T. van der
framework about antihyperglycemic medication choice. A Weijden, and A. Edwards, “Assessing the quality of decision support
prototype of this framework was developed, in which technologies using the International Patient Decision Aid Standards
guidelines and evidence studies prepared patients with instrument (IPDASi),” PLoS ONE, vol. 4, no. 3, pp. e4705, 2009.
[13] M. Coylewright, K. Shepel, A. LeBlanc, L. Pencille, E. Hess, N. Shah, V.
preliminary knowledge, and real-world EHR data were utilized M. Montori, and H. H. Ting, “Shared Decision Making in Patients with
to build a multi-label classification method to provide a Stable Coronary Artery Disease: PCI Choice,” PLoS ONE, vol. 7, no. 11,
medication recommendation list based on patients’ current pp. e49827, 2012.
[14] K. Kaiser, W. Y. Cheng, S. Jensen, M. L. Clayman, A. Thappa, F.
conditions. The applied multi-label classifier performed Schwiep, A. Chawla, J. J. Goldberger, N. Col, and J. Schein,
outstanding in severe class-imbalanced data compared to “Development of a shared decision-making tool to assist patients and
ML-kNN and RAKEL. The T2DM SDM system with decision clinicians with decisions on oral anticoagulant treatment for atrial
fibrillation,” Current Medical Research and Opinion, vol. 31, no. 12, pp.
support based on reused EHR data provides a more intuitive 2261-2272, 2015/12/02, 2015.
understanding of patient conditions and tailors knowledge [15] E. M. DeWitt, E. A. Lipstein, K. Staun, L. Scherer, J. Taylor, C. M.
about diabetes medications, leading to more effective SDM. Lannon, and W. B. Brinkman, “A178: development of tools to facilitate
shared decision making about medications for juvenile idiopathic
arthritis—a project of the pediatric rheumatology care and outcomes
improvement network,” Arthritis Rheumatol, vol. 66, pp. S232-S233,
ACKNOWLEDGMENT 2014.
[16] M. Breslin, R. J. Mullan, and V. M. Montori, “The design of a decision
The authors would like to thank the editors and the aid about diabetes medications for use during the consultation with
anonymous reviewers for their help in improving the paper. patients with type 2 diabetes,” Patient Educ Couns, vol. 73, 2008.
[17] S. Tamhane, R. Rodriguez-Gutierrez, I. Hargraves, and V. Montori,
“Shared decision-making in diabetes care,” Curr Diab Rep, vol. 15, no. 12,
REFERENCES pp. 1-10, 2015/10/12, 2015.
[1] "Diatetes Programme," Jan. 17, 2016; http://www.who.int/diabetes/en/. [18] R. J. Mullan, V. M. Montori, N. D. Shah, T. J. Christianson, S. C. Bryant,
[2] Y. Xu, L. Wang, J. He, and et al., “PRevalence and control of diabetes in G. H. Guyatt, L. I. Perestelo-Perez, R. J. Stroebel, B. P. Yawn, and V.
chinese adults,” JAMA, vol. 310, no. 9, pp. 948-959, 2013. Yapuncich, “The diabetes mellitus medication choice decision aid: a
[3] S. E. Inzucchi, R. M. Bergenstal, J. B. Buse, M. Diamant, E. Ferrannini, randomized trial,” Arch Intern Med, vol. 169, 2009.
M. Nauck, A. L. Peters, A. Tsapas, R. Wender, and D. R. Matthews, [19] A. LeBlanc, K. L. Ruud, M. E. Branda, K. Tiedje, K. R. Boehmer, L. J.
“Management of Hyperglycemia in Type 2 Diabetes, 2015: A Pencille, H. Van Houten, M. Matthews, N. D. Shah, C. R. May, B. P.
Patient-Centered Approach: Update to a Position Statement of the Yawn, and V. M. Montori, “The impact of decision aids to enhance
American Diabetes Association and the European Association for the shared decision making for diabetes (the DAD study): protocol of a
Study of Diabetes,” Diabetes Care, vol. 38, no. 1, pp. 140-149, January 1, cluster randomized trial,” BMC Health Serv Res, vol. 12, no. 1, pp. 1-9,
2015, 2015. 2012.
[4] E. S. Huang, S. E. S. Brown, B. G. Ewigman, E. C. Foley, and D. O. [20] S. E. Brossette, A. P. Sprague, J. M. Hardin, K. B. Waites, W. T. Jones,
Meltzer, “Patient Perceptions of Quality of Life With Diabetes-Related and S. A. Moser, “Association Rules and Data Mining in Hospital
Complications and Treatments,” Diabetes Care, vol. 30, no. 10, pp. Infection Control and Public Health Surveillance,” J Am Med Inform
2478-2483, October 1, 2007, 2007. Assoc, vol. 5, no. 4, pp. 373-381, 1998.
[5] A. C. Larme, and J. A. Pugh, “Attitudes of primary care providers toward [21] E. H. Shortliffe, and A. Abu-Hanna, “Clinical prognostic methods: trends
diabetes: barriers to guideline implementation,” Diabetes care, vol. 21, no. and developments,” J Biomed Inform, vol. 48, pp. 1-4, Apr, 2014.
9, pp. 1391-1396, 1998. [22] P. A. Nguyen, S. Syed-Abdul, U. Iqbal, M.-H. Hsu, C.-L. Huang, H.-C. Li,
[6] J. G. Carroll, “Crossing the Quality Chasm: A New Health System for the D. L. Clinciu, W.-S. Jian, and Y.-C. J. Li, “A Probabilistic Model for
21st Century,” Quality Management in Healthcare, vol. 10, no. 4, pp. Reducing Medication Errors,” PLoS ONE, vol. 8, no. 12, pp. e82401,
60-61, 2002. 2013.
[23] J. S. Mathias, A. Agrawal, J. Feinglass, A. J. Cooper, D. W. Baker, and A.
Choudhary2, “Development of a 5 year life expectancy index in older
2168-2194 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JBHI.2016.2614991, IEEE Journal of
Biomedical and Health Informatics
JBHI-00268-2016.R1 8
adults using predictive mining of electronic health record data,” J Am [32] G. Tsoumakas, and I. Vlahavas, "Random k-Labelsets: An Ensemble
Med Inform Assoc, vol. 20, no. e1, pp. e118-e124, 2013. Method for Multilabel Classification," Machine Learning: ECML 2007,
[24] I. Batal, D. Fradkin, J. Harrison, F. Moerchen, and M. Hauskrecht, Lecture Notes in Computer Science J. Kok, J. Koronacki, R. Mantaras, S.
“Mining recent temporal patterns for event detection in multivariate time Matwin, D. Mladenič and A. Skowron, eds., pp. 406-417: Springer Berlin
series data,” in Proceedings of the 18th ACM SIGKDD international Heidelberg, 2007.
conference on Knowledge discovery and data mining, Beijing, China, [33] Z. Huang, W. Dong, L. Ji, and H. Duan, “Outcome Prediction in Clinical
2012, pp. 280-288. Treatment Processes,” Journal of Medical Systems, vol. 40, no. 1, pp.
[25] G. Elwyn, D. Frosch, R. Thomson, N. Joseph-Williams, A. Lloyd, P. 1-13, 2015.
Kinnersley, E. Cording, D. Tomson, C. Dodd, S. Rollnick, A. Edwards, [34] K. Ng, A. Ghoting, S. R. Steinhubl, W. F. Stewart, B. Malin, and J. M.
and M. Barry, “Shared decision making: a model for clinical practice,” J Sun, “PARAMO: A PARAllel predictive MOdeling platform for
Gen Intern Med, vol. 27, no. 10, pp. 1361-1367, 2012/10/01, 2012. healthcare analytic research using electronic health records,” Journal Of
[26] Z. Min-Ling, and Z. Zhi-Hua, “A review on multi-label learning Biomedical Informatics, vol. 48, pp. 160-170, Apr, 2014.
algorithms,” IEEE Trans Knowl Data Eng, vol. 26, no. 8, pp. 1819-1837, [35] G. Elwyn, S. Laitner, A. Coulter, E. Walker, P. Watson, and R. Thomson,
2014. “Implementing shared decision making in the NHS,” BMJ, vol. 341,
[27] S. Daskalaki, I. Kopanas, and N. Avouris, “Evaluation of classifiers for an 2010-10-14 23:09:55, 2010.
uneven class distribution problem,” Appl Artif Intell, vol. 20, no. 5, pp. [36] C.-J. Ng, P.-Y. Lee, Y.-K. Lee, B.-H. Chew, J. P. Engkasan, Z.-I. Irmi,
381-417, 2006/06/01, 2006. N.-S. Hanafi, and S.-F. Tong, “An overview of patient involvement in
[28] M. R. Boutell, J. Luo, X. Shen, and C. M. Brown, “Learning multi-label healthcare decision-making: a situational analysis of the Malaysian
scene classification,” Pattern Recognition, vol. 37, no. 9, pp. 1757-1771, context,” BMC Health Serv Res, vol. 13, no. 1, pp. 1-7, 2013.
9//, 2004. [37] R. Huang, M. R. Gionfriddo, L. Zhang, A. L. Leppin, H. H. Ting, and V.
[29] N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, M. Montori, “Shared decision-making in the People’s Republic of China:
“SMOTE: synthetic minority over-sampling technique,” Journal of current status and future directions,” Patient Prefer Adherence, vol. 9, pp.
artificial intelligence research, vol. 16, pp. 321-357, 2002. 1129-1141, 08/06, 2015.
[30] L. Breiman, “Random forests,” Mach Learn, vol. 45, no. 1, pp. 5-32. [38] J. Surowiecki, and M. P. Silverman, “The wisdom of crowds,” American
[31] M.-L. Zhang, and Z.-H. Zhou, “ML-KNN: A lazy learning approach to Journal of Physics, vol. 75, no. 2, pp. 190-192, 2007.
multi-label learning,” Pattern Recognition, vol. 40, no. 7, pp. 2038-2048,
7//, 2007.
2168-2194 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.