See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/311914928

Literature Review of Patient Record Structures from the Physician’s

Perspective

Article  in  Journal of Medical Systems · February 2017

DOI: 10.1007/s10916-016-0677-0

CITATIONS READS

5 1,733

6 authors, including:

Ville Voipio Persephone Doupi

Tampere University National Institute for Health and Welfare, Finland
15 PUBLICATIONS   100 CITATIONS    61 PUBLICATIONS   546 CITATIONS   

SEE PROFILE SEE PROFILE

Hannele Hyppönen Riikka Vuokko

National Institute for Health and Welfare, Finland Ministry of Social Affairs and Health, Finland
84 PUBLICATIONS   856 CITATIONS    36 PUBLICATIONS   104 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Accidental injury prevention and safety promotion for the adult population: proposals for national action' View project

BIOINFOMED View project

All content following this page was uploaded by Persephone Doupi on 09 May 2019.

The user has requested enhancement of the downloaded file.

J Med Syst (2017) 41:29
DOI 10.1007/s10916-016-0677-0
SYSTEMS-LEVEL QUALITY IMPROVEMENT
Literature Review of Patient Record Structures
from the Physician’s Perspective
Heikki Forsvik 1,2 & Ville Voipio 1 & Johanna Lamminen 3 & Persephone Doupi 4 &
Hannele Hyppönen 4 & Riikka Vuokko 4
Received: 31 October 2016 / Accepted: 7 December 2016
# Springer Science+Business Media New York 2016
Abstract The Finnish Patient Data Repository is a nation-
wide electronic health record (EHR) system collecting patient
data from all healthcare providers. The usefulness of the large
amount of data stored in the system depends on the underlying
data structures, and thus a solid understanding of these struc-
tures is in focus in further development of the data repository.
This study seeks to improve that understanding by a system-
atic literature review. The review takes the physician's per-
spective to the use and usefulness of the data structures. The
articles included in this review study data structures intended
to be used in the actual care process. Secondary use and nurs-
ing aspects have been covered in separate reviews. After ap-
plying the predefined inclusion and exclusion criteria only 40
articles were included in the review. The research on wide-
spread systems in everyday use was especially scarce, most
studies concentrated on narrow fields. Majority of these stud-
ies were primarily developed for specialist use in secondary
care units. Most structures or applications studied were at an
early stage of development. In many applications the use of
structured data was found to improve the completeness of the
This article is part of the Topical Collection on Systems-Level Quality
Improvement
* Heikki Forsvik
heikki.forsvik@tut.fi
1
Department of Signal Processing, Tampere University of
Technology, P.O. Box 553, FI-33101 Tampere, Finland
2
General Practice/Family Medicine, University of Turku,
Turku, Finland
3
Department of Industrial Management, Tampere University of
Technology, Tampere, Finland
4
Information Services Department, National Institute for Health and
Welfare, Helsinki, Finland
documented data and facilitate its automated use. However,
there seem to be some applications where narrative text cannot
be easily replaced by structured data. Usability results regard-
ing structured representation were conflicting. The scattered
nature and paucity of research hinders the generalizability of
the findings, and from the system design or implementation
point of view the practical value of the scientific literature
reviewed is limited.
Keywords Structured data . Electronic health records .
Review . Patient records
Introduction
A large amount of information is produced by any health care
professional who participates in the care and decision-making
related to a patient. This clinical information is entered into the
health care records of the patient, and it is usually stored in an
EMR (Electronic Medical Record) or EHR (Electronic Health
Record) system. As the usefulness of the data likely depends
on the data structures used to store the data, research on these
data structures may provide valuable information for the de-
velopers and implementers of EHR systems. This study is a
systematic literature review on the EHR data structures and
their impact on the actual patient care.
The motivation of this study arises from the development
needs of the Finnish Patient Data Repository (Kanta), which is
a nationwide EHR system covering all Finnish residents1
maintained by a government agency (Social Insurance
Institution). The use of Kanta is mandated by law and it must
be used by both public and private providers [act on electronic
archiving of patient records].
1
Finnish population is approximately 5.5 million people.
29 Page 2 of 10
This study is one in a series of studies which approach the
topic from different data users' point of view. The review
protocol has been defined in an earlier study [1], and the
resulting material has then been analyzed in three different
studies. The data structures have also been studied in the con-
text of nursing [2] and secondary use [3], wheras this study
takes the clinical use viewpoint. Clinical use is defined as
using the patient information in the treatment decision-
making process, and the professionals participating in the pro-
cess producing or using the data are referred as clinicians. In
practice, this results in a physician-centric perspective.
Patient information can be entered into a patient record
system as free (narrative) text or in structured format. The
structuring methods consist of e.g. documentation standards,
forms, codes, classifications, terminologies, nomenclatures,
ontologies and conceptual models [1]. The structures can be
used in entering patient information, processing data, or re-
trieving it. Clinicians may have different roles as users of
clinical information. They produce information (by, e.g. ex-
amining a patient), consume information previously recorded,
and are users (entering and searching data) of the EHR system.
While a single clinician mostly encompasses all three roles—
they first consume earlier information, then produce new in-
formation, and finally enter the new information—the impact
of the data structures may be different in each role. For exam-
ple, strict data structures may require more analysis when
producing and entering the data than narrative text, but the
resulting data may be easier to search and consume. The sys-
tem user's point of view is closely related to the system and
user interface usability.
This study seeks to answer the following research
questions:
& Which data structures have been studied from the clini-
cians’ viewpoint?
& How do these structures impact physicians’ work, and
how has this impact been studied?
& Which effects on physicians’ work have been observed
when using structured data?
It should be noted that both the term "structured data" and
the terms EHR or EMR are not defined in the reviewed articles
in an exact way for the purposes of the review.
A specific question related to structured data is the compari-
son between narrative text (natural language) and structured da-
ta. While there is a clear difference between strict Boolean (yes/
no) data and free narrative, there is a full spectrum of data rep-
resentations between these, natural language itself has its own
structure, and highly structured information can be expressed in
natural language. This study does not attempt to create a rigorous
definition for structured data; all structures from narrative text to
Boolean data have been accepted if they have been identified as
structures in the original research.
J Med Syst (2017) 41:29
Also, the EHR/EMR aspect has been interpreted so that
both structures already in use and structures suggested to be
used have been included. Even structures based on paper
forms have been included if they could be used in an electron-
ic system without any significant modifications. The broad
approach in defining structures and EHR was taken to avoid
unnecessary elimination of original articles as well as not to
lose information due to the different approaches to EHR in
different environments.
Materials and methods
In the original literature search, there were 743 articles identi-
fied from fifteen major electronic bibliographic databases [1].
The articles underwent several exclusion rounds (see Fig. 1).
The selection criteria for the articles included in this review
are:
(1) Language (Finnish, Swedish or English for practical
reasons)
(2) Country (only higher middle or high income countries in
order to compare studies carried out in comparable health
care environment)
(3) Population (physicians)
(4) Intervention (EHR and data structures)
(5) Outcome (sufficiently clear description of the evaluation
of structured data).
The first two criteria were applied in the first round, and a
preliminary scan (Fig. 1, step II) of the remaining three criteria
was performed to divide the articles into four different cate-
gories: earlier reviews, nursing-related articles, articles
concerning secondary use of the data, and the physicians’
viewpoint. The systematic review procedure and analysis of
the reviews has been published earlier [1], as well as the anal-
ysis from the nursing viewpoint [2].
The 77 articles remaining that focused on the physicians’
viewpoint served as the starting point of this review. A final
exclusion round was performed on the articles based on non-
compliance to the criteria 3–5 outlined above (Fig. 1, step III).
Two researchers performed this analysis independently. The
articles with disagreement at this phase were re-evaluated by
both reviewers together. The evaluation criteria were
interpreted relatively loosely to avoid excluding potentially
important articles.
In practice, the exclusion was performed as a three-step
criterion-by-criterion evaluation for each article. Each exclu-
sion was classified by the first lacking criterion although some
articles could have failed to meet several criteria.
The articles excluded due to wrong population typically
described a setup where physicians and members of other
professional groups were studied together in a non-separable
J Med Syst (2017) 41:29 Page 3 of 10 29

Fig. 1 Article exclusion process

New references
from reviews
for the review

Secondary use
Reviews

Nursing
Articles
included
27 in the
10 clinical
143 review
478 322 102 40
77 1
743 15 8 13

Wrong field of study

Insufficient evaluation
Insufficient intervention
Wrong population
139

202 Excluded
63
by the
Duplicates fulltext
Excluded
by the
abstract

I Preliminary selection II Other reviews III Physician's viewpoint

way. Thus it was not possible to reliably extract the physi-
cians’ point of view from the evaluation results.
The articles excluded due to lacking intervention most
commonly reported the effect of the structure studied insuffi-
ciently. A typical example of this would be a study concen-
trating on the general usability of an EHR system with no
focus on data structures. While the data structures may have
a significant influence on usability, there are several other
major factors—such as user interface design or system respon-
siveness—which potentially overshadow the influence of the
data structure.
After the exclusion rounds the remaining publications mea-
sured the effect of the data structures from the physician’s
point of view. Even the remaining articles had some issues
with reporting the measurements or their results.
Quantitative measurements or rigorous statistical analysis
were not required for inclusion into this review, as these
criteria would have made the remaining material scant and
possibly excluded some otherwise highly relevant articles.
The final set of articles consists of slightly more than half
(n = 40) of the articles originally grouped into the physicians’
population group. The relatively small number has to be taken
into account in statistical analyses. Also, there are several
groups of articles which studied the same or almost the same
original data. This clustering may introduce statistical bias
into the results.
The articles in the final set are listed in Appendix I, and the
references to these articles are denoted by parentheses
throughout the Results and Discussion sections of this review.
Results
For quantitative analysis purposes the contents of the studies
were classified by multiple properties. The numerical results
are shown in Table 1.
Geographic distribution
The included articles were classified to belong to the country
where the actual research was carried out regardless of the
affiliation of the researchers. There are no multi-country stud-
ies, and the United States was the most active country by a
very clear margin. The two articles from Finland were pub-
lished in Finnish, others were in English.
Study design
The studies reviewed were classified by their research design.
Most studies were cross-sectional studies at one point of time
without control groups. There were a significant number of
studies utilizing randomized groups, but the randomization
applied solely to patients, not to doctors. The third group of
studies compared the situation before and after introduction of
a new system or tool.
Care level and environment
The studies were classified by the care level into primary and
secondary/tertiary care. Some interventions studied had been
either studied at both levels or were designed to be used at
both levels. The studies were also classified with a closely
related classification into practice/ambulatory or hospital/
department contexts.
A vast majority of studies were intended or carried out at
secondary/tertiary care, but there is no preferred specialty.
This emphasis on specialist care reflected clearly in the con-
text distribution, with hospital/department context being the
most common. However, there were a number of studies
which did not indicate clearly whether the structures studied
were intended for ambulatory practice or hospital contexts.
29 Page 4 of 10 J Med Syst (2017) 41:29

Table 1 Quantitative results studies where the intervention had not yet changed the care of
Classification criteria / categories n the patients. This was the most common phase of intervention
(N = 22, 55% of the articles) (3, 5, 12, 14–16, 18–19, 21, 23–
Geographic distribution 24, 28–34, 37–40).
North America 26 A study was regarded as a clinical testing, if the structure
Canada 3
United States 23 had an impact on patient care process but was not yet in rou-
Europe 11 tine use. The main purpose of these trials was to produce new
Germany 2 information on the structure or application (N = 12, 30% of the
Finland 2
Netherlands 4 articles) (1–2, 4, 7–9, 11, 20, 22, 26–27, 35).
United Kingdom 1 There were also a small number of studies which concen-
Norway 2 trated on evaluating patient record structures in routine use.
Asia & Pacific 2
Hong Kong 1 These studies had a narrow scope and they were technologi-
New Zealand 1 cally simple, e.g. assessing a new data entry form. (N = 6, 15%
South America 1 of the articles) (6, 10, 13, 17, 25, 36) In general, the techno-
Argentina 1
Study design logical complexity of the interventions decreased from early
Single cross-section 23 phase to routine use.
Randomized 12
Before/after 5
Care level
Primary health care 3 Structures and applications studied
Secondary/tertiary health care 33
Both 4
The articles described studies with various structuring
Environment
Practice/ambulatory 22 methods from automated coding systems to automatic natural
Hospital/department 9 language processing with free text. The spectrum of structures
Not available 9
varied from introducing a new data entry form (e.g. 4, 7, 18,
Phase of study
Early phase 22 27) to a fully automated natural language processing neural
Clinical trial 12 network diagnostic system (e.g. 5). Due to the wide spectrum
Routine use 6 of interventions it is not possible to make an unambiguous
Structures and applications studied
Codes/taxonomy 27 classification of structures assessed or applied, and the border
Documentation standards 22 between technical realization and intervention is not always
Narrative text 15 completely clear. Also, the research may have exploited some
Changes to paper forms 8
Data collection methods structures which have not been studied or are insignificant
Extraction of EHR data 28 from the study setup point of view. Several studies evaluated
Questionnaires 14 more than one structure, and thus the classification is per
Interviews 4
Observations 4
structure, not per study. For example, an intervention may be
Data sources the introduction of a new data entry form (local documenta-
Patient (health) data 36 tion standard) which uses certain codes.
Clinicians 24
Indicators
The most common structures studied were codes and tax-
Data quality 35 onomies, especially diagnostic coding and problem lists,
Usability 12 which were assessed extensively (N = 26, 65%) (1–2, 4, 6–
Time spent 9
9, 12, 14–17, 19–20, 22–26, 28, 30–32, 35, 39–40). Typically,
Quality of service 3
Impact of structured data the suitability of a coding into a specific environment was
Completeness vs. narrative data 10 studied.
Time saving 3 Another common subject was documentation standards
Impacts not evaluated 27
and forms (N = 22, 55%) (1–2, 4, 7–11, 13–14, 17–18, 24,
The total number of publications reviewed is 40, and a single publication 26–28, 33–38). There were several articles where the interven-
may belong to several categories at the same time. Only the categories tion was a paper form (N = 8, 20%) (4, 7, 10, 13, 17, 27, 35–
relevant to this review are shown. 36). While these studies were not EHR-based, their outcomes
can be assumed to be relevant also with EHRs. Although all
Phase of intervention studies of common documentation standards and forms were
included into this category, widespread international standards
The phase of intervention was classified into early phase, clin- were not commonly studied. The same applies to conceptual
ical testing, and established use to find out whether the struc- models; they —especially HL7—were mentioned in the arti-
tures were already in use. Early phase of intervention included cles but were not the actual subject of the research.
J Med Syst (2017) 41:29
Free text processing was a relatively common subject of
evaluation (N = 14, 35%) (3, 5, 6, 8, 12, 18, 20–25, 30–31).
The purpose of these studies was to assess the methods of
(automated) structuring of free text records. The methods var-
ied from simple keyword search to sophisticated natural lan-
guage processing applications.
Data collection methods, sources, and indicators
By far the most common data collection method was extrac-
tion of data from an existing EHR database. Patient informa-
tion was used in almost all studies in some form, and it was the
sole information source in several articles (N = 12, 30%) (2, 4,
7, 11, 13, 16, 20–21, 24, 26–27, 32). With this kind of data
collection the number of entries may be large (in some cases
n > 100.000). However, the lack of wide scale machine-usable
long-time records restricts the large-scale studies into cross-
sectional studies. All studies utilizing pre-existing patient in-
formation have been counted into this category regardless of
the terminology used.
Questionnaires and interviews were widely used in 16 ar-
ticles (40%) (1, 6, 8–9, 18–19, 25, 28, 30–31, 33–38). The
questionnaires were targeted at clinicians, patient-related pa-
rameters, e.g. patient satisfaction, were not measured. There
were also some studies using observational data. Several stud-
ies combined different data collection methods.
Indicators
The evaluation methods applied to the interventions varied
significantly, and in some cases the methods were only de-
scribed on a superficial level. In several studies the evaluation
did not focus on the data structures but on a wider interven-
tion, possibly including other factors, such as the actual real-
ization of the user interface. Thus, the classification of evalu-
ation methods has a large error margin.
The quality of information was studied in almost all stud-
ies. The most common way to measure the quality (N = 11,
28% of articles) was counting the occurrence of predefined
data items (2–4, 8, 13–14, 17, 26–28, 35). Another clear group
of studies researched the usability of data structures (N = 11,
28%). These studies used tools developed for generic usability
studies, and questionnaires played an important role (1, 6, 8–
9, 18, 25, 28, 34–36, 38). Efficiency of performing a certain
system function was measured in several studies, mostly as
time spent filling in a single form or searching a data item
(N = 9, 23%) (1, 6, 9, 14, 18, 28, 33–35).
Usability was evaluated in a number of studies, but the
most common question was related to user acceptance of
structured data and the associated data entry arrangement.
The usability was mostly evaluated as acceptable or as an
improvement over some earlier methods (1, 6, 9, 18, 25, 34)
but some studies acknowledged that the usability may have
Page 5 of 10 29
been affected by other factors than the underlying data struc-
ture, mostly the user interface (6, 25–26, 28, 38). Several
studies suggested that structured data is easy to process (9,
31, 38, 40), but coding and entering structured data is slower
for the user, and thus it may be perceived more cumbersome
(6, 28), and some clinicians preferred free text data entry (24,
40). Slower data entry was explained, e.g., by learning effects
and increase in the amount of data (28). However, there were
also studies which found that structured data is faster to enter
(14, 18) and makes work processes easier (36). Two articles
by the same author described the effect of data organization on
the data retrieval time; relatively small changes in the data
structure may have a significant impact on the retrieval time
(33, 34).
Some studies concluded that data structures and data entry
processes had a significant impact on the thought processes of
the physician (31, 33, 37).
Impacts of patient record structures on clinical work
The review material did not allow quantitative analysis on the
impact of the interventions on clinical work, but there were
some qualitative results or conclusions which occurred
repeatedly.
The most common positive impact (N = 9, 23%) was relat-
ed to the completeness of the information (4, 6, 8, 13–14, 17,
27–28, 35). Structured data was mentioned to be more com-
plete than narratives. However, the completeness was usually
defined by verifying whether certain pre-defined data items
exist; and this definition does not exclude the possibility that
narratives would contain more clinically relevant information
than structured data. There were no studies which would have
measured and shown an improvement in the care outcome due
to the improved completeness of data. There was a single
study which did not find significant improvement in informa-
tion completeness when narrative text was replaced by a form
(27).
Publication years
The distribution of the articles' publication years is shown in
Fig. 2. The data shows an increase of research activity around
2005, but the activity has not increased after that.
Discussion
Review material – limitations of the study
The number of available studies relevant to the focus of this
article was small, and it seems that despite the significance of
structured data in clinical use the impact of structured patient
records has not been extensively researched. An alternative
29 Page 6 of 10
Fig. 2 Publication year
distribution of the articles
included into the review
explanation to this observation cannot be ruled out— i.e. that
the search process may have had some significant limitation
which has led to the omission of a large number of relevant
articles. However, we do not find this explanation likely, as the
number of articles in the raw material before the exclusion
round was relatively large, and a strict protocol for systematic
reviews was followed.
The wide variability of material included is the most obvi-
ous weakness of this review. It is difficult to produce statisti-
cally significant quantitative results, and a lot of important
information seems to lie in small comments or discussion in
the articles reviewed. This leads to somewhat subjective re-
sults and interpretations, which is clearly a weak point from
the systematic review perspective.
Most studies included in the review were narrow either by
their scope or by their data collection, which is also acknowl-
edged by the authors. Majority of research consisted of early
phase projects where the intervention had not yet changed or
impacted the patient care. Care outcome was not studied com-
monly, i.e. an important impact area is not covered.
Time trends
An interesting but somewhat disturbing observation is that the
publication dates do not reveal an increasing trend in research
activities on the field. Due to the low number of publications
this observation may not be statistically significant, but it de-
serves a follow-up later on. While the material of the study
does not include articles from the last years, a simple literature
search with the same keywords does not reveal any significant
increase in the research activity.
One possible explanation behind the lack of increasing ac-
tivity on the field may be explained by the widespread com-
mercialization of EHR systems. Commercial motivations may
diminish the willingness to subject these systems to scientific
system research.
There does not seem to be any strong temporal trend on the
research areas, either. For example, natural language has been
studied all the time. The only code or taxonomy which seems
to be persistent is the ICD classification. Other codings and
taxonomies have quite random occurrences.
J Med Syst (2017) 41:29
The complexity of the structures and applications was ob-
served to decrease from early phase to established use. This
could easily be explained by the advent of technology on a
young field; at any given time there is more advanced tech-
nology at the earlier phases. However, this should be visible in
the publications describing established technology, but no
such trend is visible. Either the technologically challenging
applications do not make it into established use, or they dis-
appear form the research field e.g. due to commercialization.
Missing research areas
Due to the scattered nature of research available, it is impor-
tant to identify the missing research areas. Possibly the most
significant blind spot in the material was the lack of research
on the use of data structures in wide-scope EHR systems in
operational use. This is likely to be explained by several prac-
tical reasons; EHR systems in operational use form a critical
link in the care chain, and that, together with their very high
commercial value, may limit the possibility of doing experi-
mental research on the different data structures. Furthermore,
it is very challenging to design a study which would not mea-
sure all usability factors together but only the impact of the
data structures.
Usability is not an unimportant factor when looking at the
data structures form the physician's perspective. This is one of
the key differences between secondary (research) data use and
clinical use. In secondary use the quality and quantity of the
data collected is the driving factor, whereas in clinical use
usability has an important role. The data structures and asso-
ciated data entry systems seem to have an impact on the actual
work processes, but very few articles studied this aspect.
The outcome of the health care process was seldom mea-
sured. As the research concentrates on relatively small details
of the care chain, the patient does not take an important role.
The emerging trend of personal health records is not visible in
the articles. This may be partially explained by the age of
articles, but even taking that into account the patient perspec-
tive is lacking.
There were very few studies on the use of data structures in
primary healthcare; the general rule seems to be that the more
J Med Syst (2017) 41:29
specialized the field, the more research on data structures there
is. One underlying reason may be the amount of information.
As primary healthcare combines all specialties, the number
and variety of codes and structures may become too large to
be mastered by a single practitioner. According to our study,
another explanation may be that while the coding and taxon-
omy research works towards unified data models, only diag-
nostic codes are mature enough to be used in primary
healthcare. On the other hand, this interpretation raises a ques-
tion whether the main factor is organizational or technical
maturity, as organizations are already accustomed to using
diagnostic codes.
An attempt was made to isolate the data structures and
contexts, but the descriptions of all relevant factors – user
interface, the EHR, the work process, the people using the
EHR, etc. – were insufficient, and thus there may be a signif-
icant influence from the context. It may even be that studying
data structures separately from the context is not practical,
because the usefulness of a data structure depends on how
well it fits the use context.
One of the aims of this review was to produce relevant
information for designers of a national multi-disciplinary
EHR-archiving system. While it is not known to which extent
this review fulfils that objective, it may even be that the con-
tingent nature of EHR makes producing this kind of informa-
tion very challenging.
Narrative text vs. structured data
An important question from the clinical viewpoint related to
data structures concerns the use of narrative text in patient
records – and whether it is needed at all. The comparison of
impacts of narrative text versus structured data gave contra-
dictory results.
The contradictory results between studies suggesting that
structured data is faster to enter, easy to process and makes
work processes easier, and studies suggesting that coding and
entering structured data is slower for the user and clinicians
prefer free text data entry may be explained by several factors.
Structured data may by its nature require more cognitive effort
when it is entered (33), but it is easier to process. There is also
some evidence that the perceived difficulties are in some cases
explained by the lack of familiarity with the structure or cod-
ing system (39). Structured data also offered a possibility of
creating different views into the same data (31) according to
the preferences of the user.
The feasibility of replacing narratives with structured data
seems to depend on the nature of the information to be en-
tered. Narrative text was most difficult to be replaced when it
described thought processes (8). Combining the two data
types could be beneficial; allowing free text entry along with
structured text may increase user acceptance (6, 15). There are
also arguments against this combination; the same flexibility
Page 7 of 10 29
may be obtained by building flexibility into the structures
(16).
The use of data structures guided the data entry process to a
certain extent, i.e. the information entered as narrative text did
not necessarily have the same information content as informa-
tion entered in structured form (28). Thus replacing narrative
text with structured data as the data entry method carries the
risk of losing some information which has not been defined
beforehand but eliminates the risk of accidentally omitting
some predefined data items. There was some evidence sug-
gesting that the amount of narrative text should be minimized
to make the information as uniform as possible (16).
The gap between narrative text and structured data may
potentially be bridged by using natural language processing
(NLP) algorithms which enable processing the text with, e.g.,
automated classifiers or indexing. Our conclusion is that NLP-
based tools may be useful in interpreting narrative documents
in situations when it is not possible or reasonable for the user
to read them, but at its current state the technology is not
mature enough to be used in unsupervised conversion be-
tween narrative text and structured data for clinical use. It is,
however, important to note that this conclusion arises from the
small error tolerance of clinical data, and NLP may prove to be
useful in, e.g., search tools. Also, it is possible that the tech-
nology might be or might very soon become useful in a
broader context, but the obstacles are more in proving the
usefulness of the technology and in external factors (organi-
zational and legislative issues).
The review material did not include any studies focusing
on special issues in international or multi-lingual environ-
ments. Narrative text most often requires manual translation,
whereas structured data does not have inherent language-de-
pendency. However, in practice there are some language-de-
pendencies, and full interchangeability requires the use of
same or sufficiently similar data structures across the organi-
zations, as automatic translation between different data struc-
tures may not be possible. Thus this benefit associated with
structured data may not be easy to exploit in an international
context where all parties have their local or national data
structures.
Data structures in wider context
Focusing on the data structures used in EHR systems may be
somewhat misleading in wider context. The fit of data struc-
tures and/or EHR systems was not studied within the review
material, but it seems reasonable to suggest that the impact of
a data structure is dependent on several external factors, at
least work processes and other data structures used.
However, while finding the optimal solutions in large ap-
plications may be difficult, there was a lot of evidence that
even small interventions may be effective, for example using
29 Page 8 of 10
unified naming schemes for documents and sections (9, 33,
34).
Conclusions
As noted in the previous section, there is a scarcity of high
quality, quantitative research into the area. Thus, the main
conclusion of the study is that there is a clear need of further
research into the area. For the same reasons it is difficult to
draw any very strong conclusions.
Regarding the debate between narrative text and structured
data, the material seems to support the idea that structured data
is often more useful and more complete than unstructured
data. Furthermore, this usability is not limited to automated
use or secondary users; structure may make clinicians' con-
sumption of information easier. Free narrative text is strongest
in applications where descriptions of thought processes or
expressions of uncertainty and probabilities are required.
However, it should be noted that there is a continuum between
simple structured data and narrative text; natural language can
be used in a structured way, and there are very flexible data
structures.
The research suggests that already small and trivial inter-
ventions may have a significant impact on the outcome or
flow of care processes. Examples of these simple but effective
interventions are common document naming schemes and
division of a document under predefined sections.
While the applicability of coding systems in different en-
vironments has been researched in several studies, it is diffi-
cult to draw clear conclusions from the results. These studies
tend to observe the usability of the codes in a very specific
environment or application area, which limits the generaliz-
ability of the results outside of this narrow scope.
Studying the usability of structures in documentation and
use of patient information is very relevant in the EHR context,
yet the most important research result seems to be the sugges-
tion that structures are one—but not the only or dominant—
factor affecting EHR usability. The coupling between struc-
tures and user interfaces is so tight that their impact is difficult
to tell apart.
The subjective user experience is an important part of us-
ability, and thus the preferences and skills of the users have an
important impact on the outcome. One branch of usability
research concentrates on users' cognitive processes. These
studies show that the individual differences in clinical
decision-making processes between users are significant.
The most important aim of this review was to provide prac-
tical information for large scale EHR system designers on the
different data structures. This goal was not reached, as the
review does not offer much information beyond what is gen-
erally known in the field. Usability is important, especially
when studied in a wider context, but probably the most
J Med Syst (2017) 41:29
effective and important direction for new research on the field
would be to take a top-down approach into the work processes
and the match of data structures and data entry or display
systems to those processes.
Appendix A – List of articles reviewed
1. Aro S, Hagman E, Vohlonen I. Kansainvälinen
perusterveydenhuollon luokitusjärjestelmää lääkärissä
käyntien tarkastelussa. Suomen lääkärilehti - Finlands
läkartidning. 1986;41(32):3050-6.
2. Aronsky, D., Haug, PJ. Assessing the quality of clinical
data in a computer-based record for calculating the
Pneumonia Severity Index. Journal of the American
Medical Informatics Association. 2000 JAN-
FEB;7(1):55-65.
3. Aronsky, D., Kasworm, E., Jacobson, J. A., Haug, P. J.
and Dean, N. C. Electronic screening of dictated reports
to identify patients with do-not-resuscitate status.
Journal of the American Medical Informatics
Association. 2004 Sep-Oct;11(5):403-9.
4. Bar-on, M. E., Zanga, J. R. Child abuse: a model for the
use of structured clinical forms. Pediatrics. 1996
Sep;98(3 Pt 1):429-33.
5. Carl-F., Bassøe. Automated diagnoses from clinical nar-
ratives: A medical system based on computerized med-
ical records, natural language processing, and neural net-
work technology. Neural Networks. 1995;8(2):313-9.
6. Bell, D. S., Greenes, R. A. Evaluation of UltraSTAR:
performance of a collaborative structured data entry sys-
tem. Proceedings - the Annual Symposium on Computer
Applications in Medical Care. 1994:216-22.
7. Biffl, W. L., Harrington, D. T. and Cioffi, W. G.
Implementation of a tertiary trauma survey decreases
missed injuries. Journal of Trauma-Injury Infection &
Critical Care. 2003 discussion 43-4; Jan;54(1):38-43.
8. Brown, M. L., Quinonez, L. G. and Schaff, H. V. A pilot
study of electronic cardiovascular operative notes: qual-
itative assessment and challenges in implementation. J
Am Coll Surg. 2010 Feb. 210(2):178-84.
9. Brown, S. H., Lincoln, M., Hardenbrook, S., et al.
Derivation and evaluation of a document-naming no-
menclature. Journal of the American Medical
Informatics Association. 2001 Jul-Aug. 8(4):379-90.
10. Cheng, G., Wong, H. F., Chan, A. and Chui, C. H. The
effects of a self-educating blood component request
form and enforcements of transfusion guidelines on
FFP and platelet usage. Queen Mary Hospital, Hong
Kong. British Committee for Standards in Hematology
(BCSH). 1996
J Med Syst (2017) 41:29
11. Co, John P. T., Johnson, Sarah A., Poon, Eric G., et al.
Electronic Health Record Decision Support and Quality
of Care for Children With ADHD. Pediatrics. 2010
08;126(2):239-46.
12. D'Avolio, LW, Litwin, M. S., Rogers, SO, J. and Bui,
AAT. Facilitating clinical outcomes assessment through
the automated identification of quality measures for
prostate cancer surgery. J Am Med Inform Assoc. 2008
2008;15(3):341-8.
13. Geist, S. M., Geist, J. R. Improvement in medical con-
sultation responses with a structured request form. J
Dent Educ. 2008 May;72(5):553-61.
14. Hey, C., Pluschinski, P., Stanschus, S., et al. A
documentation system to save time and ensure proper
application of the fiberoptic endoscopic evaluation of
swallowing. (FEES). Folia Phoniatrica et Logopedica.
2011;63(4):201-8.
15. Krall, M. A., Chin, H., Dworkin, L., Gabriel, K. and
Wong, R. Improving clinician acceptance and use of
computerized documentation of coded diagnosis. Am J
Manag Care. 1997 Apr;3(4):597-601.
16. Los, R. K., van Ginneken, A. M., Roukema, J., Moll, H.
A. and van der Lei, J. Why are structured data different?
Relating differences in data representation to the ratio-
nale of OpenSDE. Medical Informatics & the Internet in
Medicine. 2005 Dec;30(4):267-76.
17. Marco, Alan P., Buchman, Debra and Lancz, Colleen.
Influence of form structure on the anesthesia preopera-
tive evaluation. J Clin Anesth. 2003 9;15(6):411-7.
18. Marill, K. A., Gauharou, E. S., Nelson, B. K., Peterson,
M. A., Curtis, R. L. and Gonzalez, M. R. Prospective,
randomized trial of template-assisted versus undirected
written recording of physician records in the emergency
department. Ann Emerg Med. 1999 May; 33(5):500-9.
19. Mayo, N. E., Poissant, L., Ahmed, S., et al. Incorporating
the International Classification of Functioning, Disability,
and Health (ICF) into an electronic health record to create
indicators of function: proof of concept using the SF-12.
Journal of the American Medical Informatics Association.
2004 Nov-Dec;11(6):514-22.
20. Meystre, S. M., Haug, P. J. Randomized controlled trial
of an automated problem list with improved sensitivity.
Int J Med Inf. 2008 Sep; 77(9):602-12.
21. Meystre, S. M., Haug, P. J. Comparing natural language
processing tools to extract medical problems from nar-
rative text. AMIA …Annual Symposium Proceedings/
AMIA Symposium: 525-9, 2005.
22. Meystre, S., Haug, P. Improving the sensitivity of the prob-
lem list in an intensive care unit by using natural language
processing. AMIA…Annual Symposium proceedings /
AMIA Symposium.AMIA Symposium.554-8p.2006.
23. Meystre, S. Haug, PJ. Natural language processing to
extract medical problems from electronic clinical
Page 9 of 10 29
documents: Performance evaluation. J Biomed Inform.
2006 12; 39(6):589-99.
24. Mikkelsen, G., Aasly, J. Manual semantic tagging to
improve access to information in narrative electron-
ic medical records. Int J Med Inf. 2002
APR;65(1):17-29.
25. Navas, H., Osornio, A. L., Baum, A., Gomez, A., Luna,
D. and de Quiros, F. G. Creation and evaluation of a
terminology server for the interactive coding of dis-
charge summaries. Studies in Health Technology &
Informatics. 2007;129(Pt 1):650-4.
26. Niinimäki,, J. et al. Strukturoitu teksti korvaamaan
kertovat sairauskertomustekstit ja niiden luokittelut?
Suomen lääkärilehti - Finlands läkartidning.
1996;51(5):453-7.
27. O'Connor, A. E., Finnel, L. and Reid, J. Do preformatted
charts improve doctors documentation in a rural hospital
emergency department? A prospective trial. N Z Med J.
2001 Oct 12;114(1141):443-4.
28. Roukema, J., Los, R. K., Bleeker, S. E., van Ginneken,
AM, J and Moll, H. A. Paper versus computer: feasibil-
ity of an electronic medical record in general pediatrics.
Pediatrics. 2006;117(1):15-21.
29. Schleyer, T., Spallek, H. and Hernandez, P. A qualitative
investigation of the content of dental paper-based and
computer-based patient record formats. J Am Med
Inform Assoc. 2007 2007;14(4):515-26.
30. Shapiro, J. S., Bakken, S., Hyun, S., Melton, G. B.,
Schlegel, C. and Johnson, S. B. Document ontology:
supporting narrative documents in electronic health re-
cords. AMIA …Annual Symposium Proceedings/
AMIA Symposium.:684-8, 2005.
31. Sharda, P., Das, A. K., Cohen, T. A. and Patel, V.
Customizing clinical narratives for the electronic medi-
cal record interface using cognitive methods. Int J Med
Inf. 2006 May;75(5):346-68.
32. Spangler, W. E., May, J. H., Strum, D. P. and Vargas, L.
G. A data mining approach to characterizing medical
code usage patterns. J Med Syst. 2002 Jun;26(3):255-75.
33. Tange, H. J., Schouten, H. C., Kester, A. D. and Hasman,
A. The granularity of medical narratives and its effect on
the speed and completeness of information retrieval.
Journal of the American Medical Informatics
Association. 1998 Nov-Dec;5(6):571-82.
34. Tange, HJ. Consultation of medical narratives in the
electronic medical record. Methods Inf Med. 1999
DEC;38(4-5):289-93.
35. Thomas, K., Emberton, M. and Reeves, B. The use of a
structured form during urology out-patient consultations
– a randomised controlled trial. Methods Inf Med.
2005;44(5):609-15.
36. Treece, P. D., Engelberg, R. A., Crowley, L., et al.
Evaluation of a standardized order form for the
 29 Page 10 of 10
withdrawal of life support in the intensive care unit. Crit
Care Med. 2004 May;32(5):1141-8.
37. Van Vleck, T. T., Stein, D. M., Stetson, P. D. and
Johnson, S. B. Assessing data relevance for automated
generation of a clinical summary. AMIA …Annual
Symposium Proceedings/AMIA Symposium.:761-5,
2007.
38. Van Walraven, C., Duke, S. M., Weinberg, A. L. and
Wells, P. S. Standardized or narrative discharge summa-
ries. Which do family physicians prefer? Canadian
Family Physician. 1998 Jan;44:62-9.
39. Wasserman, H., Wang, J. An applied evaluation of
SNOMED CT as a clinical vocabulary for the comput-
erized diagnosis and problem list. AMIA ..Annual
Symposium Proceedings/AMIA Symposium.:699-703,
2003.
40. White, J. M., Kalenderian, E., Stark, P. C., Ramoni, R.
L., Vaderhobli, R. and Walji, M. F. Evaluating a dental
View publication stats
J Med Syst (2017) 41:29
diagnostic terminology in an electronic health record. J
Dent Educ. 2011 May;75(5):605-15.
References
1. Hyppönen, H., Saranto, K., Vuokko, R., Mäkelä-Bengs, P., Doupi,
P., Lindqvist, M., and Mäkelä, M., Impacts of structuring the elec-
tronic health record: a systematic review protocol and results of pre-
vious reviews. Int. J. Med. Inform. 83(3):159–69, 2014.
2. Saranto, K., Kinnunen, U., Kivekäs, E., Lappalainen, A., Liljamo, P.,
Rajalahti, E., and Hyppönen, H., Impacts of structuring nursing re-
cords: a systematic review. Scand. J. Caring Sci. 28(4):629–47,
2014.
3. Vuokko, R., Mäkelä-Bengs, P., Hyppönen, H., Lindqvist, M., and
Doupi, P. Impacts of structuring the electronic health record: results
of a systematic literature review from the perspective of secondary
use of patient data, submitted.