2016 IEEE 29th International Symposium on Computer-Based Medical Systems
Quality Evaluation of Poison Control Information
Systems: A Case Study of the DATATOX System
João M. Alves, Christiane G. von Wangenheim, Aldo von Wangenheim
Postgraduate Program in Computer Science
Federal University of Santa Catarina – UFSC
Florianópolis, Brazil
joao@telemedicina.ufsc.br, gresse@inf.ufsc.br, aldo.vw@ufsc.br
Marlene Zannin
Pathology Department
Universidade Federal de Santa Catarina – UFSC
Florianópolis, Brazil
marlenezannin@gmail.com
Abstract—Poison Control Centres (PCCs) provide information
and advise about poisoning, poisoning case management, and
toxicovigilance activities. They are supported by information
systems that help physicians and toxicologists to attend cases of
patient exposure to poisonous agents. Still, there is no consensus
on how to evaluate software quality of such systems. This
work performs a case study applying a customized software
quality evaluation model – the AdEQUATE model – in order
to assess the software quality of a poison control information
system widely used in Brazil – the DATATOX information
system. Analysing the specificities of DATATOX we customized
AdEQUATE, a model for evaluating the quality of telemedicine
systems based on the ISO/IEC 25010 standard through the use of
a questionnaire. The questionnaire has been answered by 49 end
users, and the acquired data were analysed employing descriptive
statistics. Overall, the DATATOX information system has been
perceived as having a good quality, specifically for characteristics
like effectiveness, efficiency, and satisfaction. Nevertheless, few
characteristics have been considered of low quality, including
reliability, compatibility, and age. As a complement, the work
attest the feasibility on using AdEQUATE’s questionnaires as an
evaluation tool for software quality evaluation.
Keywords— software quality evaluation; poison control centres
information systems; case study;
I. I NTRODUCTION
Poisoning has caused anguish and adversities throughout
history. The literature points out that from the ergot epidemics
of the Middle Ages up to contemporary industrial disasters,
these plagues have had great political, economical, social, and
environmental implications – particularly within the past 100
years – as the number of toxins and potential toxins handled
in everyday situations has risen dramatically [1]. Yearly,
poisoning accounts for about 170,000 hospital admissions in
the NHS (National Health Service) of the United Kingdom, a
considerable workload for the health service staff, especially in
hospital emergency departments and medical admission units
[2]. Information about poisoning in Brazil (based on reports
from the States of Santa Catarina [3] and Rio Grande do
2372-9198/16 $31.00 © 2016 IEEE
DOI 10.1109/CBMS.2016.53
Alexandre Savaris
Telemedicine Laboratory
Brazilian Institute for Digital Convergence – INCoD
Florianópolis, Brazil
savaris@telemedicina.ufsc.br
Danielle B. L. Albino, Marisete C. Resener
Santa Catarina Toxicology Information Centre – CIT/SC
Brazilian Association of Toxicology Information and
Assistance Centres – ABRACIT
Florianópolis, Brazil
danialb@terra.com.br, mcresener@yahoo.com.br
Sul [4]) shows that the number of reported intoxications has
risen 46% from 2005 to 2013, with over 147,000 cases of
intoxication by drugs or venomous animals being registered
during that period. From the registered cases 0.31% ended in
death, and 0.59% resulted in sequelae for the patients.
Poison Control Centres (PCCs) have the responsibility to
provide toxicological information and advice, management of
poisoning cases, provision of laboratory analytical services,
surveillance activities, research, education, and training in the
prevention of poisoning [5]. Countries such as the United
States [6] and the British Isles [7] maintain poisoning infor-
mation centres that provide toxicological advisory services 24
hours a day. Most of those services rely on telephone inter-
action, but some PCCs have on-line interaction systems, such
as the Web Poison Control website [8], allowing information
requesters to get help using the Internet.
In this context, Teletoxicology is a specialization of
telemedicine that provides toxicology services at a distance,
which can be used in the absence of immediately available
toxicology specialists [9]. Although easily defined, there is no
consensus in the literature about the objectives to be achieved
and the main functionalities to be provided by teletoxicology
systems: TOXBASE [10] and the American National Poison
Data System [11], for instance, act respectively as a poison
information database for health care professionals and as a
record of cases reporting the exposure of patients to poisonous
agents. Despite its final aims, such systems manage informa-
tion that has a direct impact on patient’s health; therefore, it
is important to maintain a minimal level of quality to prevent
errors that could impose risks to patients.
There exist quality guidelines that could be applied to
teletoxicology systems, as the guideline for poison control
centres of the World Health Organization [5], and the Practice
Guidelines for Live, on Demand Primary and Urgent Care of
the American Telemedicine Association [12]. However, these
guidelines usually do not consider software quality as a whole,
30
limiting themselves to proposals on how a specific quality
characteristic should be addressed by systems. On the other
hand, over the years a number of software quality evaluation
models and standards have been proposed to support quality
control of software systems [13]. Yet, models like ISO/IEC
25010 [14] are generic and, thus, may not fully address the
specific needs of teletoxicology systems. In order to ensure
that a specific product meets user and customer needs, it is
necessary to customize the quality evaluation model to the
context at hand [15].
A customization approach based on the ISO/IEC 25010
standard is AdEQUATE (questionnAire for Evaluation of
QUAlity in TElemedicine systems) [16]. AdEQUATE is
an evaluation model built considering the specificities of
telemedicine systems, decomposing an evaluation goal repres-
ented as a set of analysis questions into metrics, being exerted
by a questionnaire in order to allow data analysis from the
end user’s perception.
In this work we focus on evaluating the software quality
of DATATOX, a teletoxicology information system supported
by the Brazilian Association of Toxicology Information and
Assistance Centres (ABRACIT) used in nine States by toxico-
logy experts to aid poisoned victims, manage poisoning cases,
and provide statistics to the Brazilian National Health Sur-
veillance Agency. The evaluation is made from the viewpoint
of end users – on-duty toxicologists, medical students, and
physicians – employing the AdEQUATE model. We aim, also,
to attest the feasibility of systematically generate customized,
ISO/IEC 25010-based software quality models able to evaluate
in detail teletoxicology systems, addressing its specificities
while maintaining the generality and software-quality-as-a-
whole approach of the referred standard.
In Section II we provide a background on teletoxicology
systems and software quality evaluation. In Section III we
describe the customization process performed on AdEQUATE
and its application through questionnaires. The collected data
is analysed and discussed in Sections IV and V, guiding the
conclusions about the feasibility of the evaluation approach,
which is presented in Section VI.
II. BACKGROUND AND R ELATED W ORKS
A. Toxicology and Poison Information Systems
Clinical toxicology concerns the investigation, diagnosis and
management of suspected poisoning cases [17], contributing
for the reduction of rates related to drug abuse [18] and in
helping victims of poisoning by several agents, such as drugs,
pesticides, bites and envenomations, plants, and chemicals [6].
In this context, teletoxicology systems are information sys-
tems that support the referred practice over distance. Despite
its importance, there are few works available describing its
architecture, workflow, and common functionalities. In order
to provide a consistent background in teletoxicology systems,
we conducted a systematic literature review (SLR) following
the procedures defined by Kitchenham [19]. The scope of our
SLR is limited on literature about information systems in a
31
toxicology environment, gathering references from three sets
of repositories:
Set #1: Sagepub1 , IEEEXplore2 , Springer Link3 , ACM
Digital Library4 , and MEDLINE5 . The search was
performed using the following well-formed string:
"((toxicology AND information system) OR
(teletoxicology AND information system))";
Set #2: Journal of Telemedicine and e-Health6 , International
Journal of Telemedicine and Applications7 , International
Journal of Telemedicine and Clinical Practices8 , and
International Journal of E-Health and Medical Commu-
nications9 . The search included all references related to
toxicology;
Set #3: Journal of Medical Toxicology10 , Toxicology11 , and
Trends in Pharmacological Sciences12 . The search in-
cluded all references related to information systems.
We considered all state-of-the-art references that met the
following criteria: a) Publication in English; b) focus on
information systems for toxicology; and c) publication date
between January, 2011 and December, 2015 . References that
do not describe an information system or quality aspects of
an information system were excluded from the result set. The
initial search executed in November, 2015 resulted in a total
of 848 references. By analysing the title and keywords and
applying the inclusion and exclusion criteria, we identified 38
potentially relevant articles. After a careful reading of its full
content, we considered that 11 articles match all criteria. Table
I summarizes the SLR result.
According to the classification, three references have focus
on teleconsultation (i.e. the telemonitoring intervention includ-
ing patient–caregiver asynchronous or synchronous commu-
nication to connect caregivers and one or more patients simul-
taneously [20]). Information databases, in turn, correspond to
the most common type of teletoxicology system, addressing
the toxic effects of medications and hazardous chemicals –
including household products – providing information about
general and environmental health issues [21].
Software tools supporting the toxicology practice are cited
by one reference, and poisoning control systems – focus of the
current work – are present in two references. None of those
works focused on the evaluation of its software quality.
Although not explicitly defined, [25], [28], [8], [32], [33],
[34], [35] list three categories of stakeholders for teletoxi-
cology systems: Requesters, attendants, and physicians. Re-
1 http://online.sagepub.com/
2 http://ieeexplore.ieee.org/Xplore/home.jsp
3 http://link.springer.com/
4 http://dl.acm.org/
5 http://www.ncbi.nlm.nih.gov/pubmed
6 http://online.liebertpub.com/loi/TMJ
7 http://www.hindawi.com/journals/ijta/
8 http://www.inderscience.com/jhome.php?jcode=ijtmcp
9 http://www.igi-global.com/journal/international-journal-health-medical-
communications
10 http://link.springer.com/journal/13181
11 http://www.journals.elsevier.com/toxicology/
12 http://www.cell.com/trends/pharmacological-sciences/home
 Table I 1) The DATATOX information system: DATATOX is an
R ELATED TELETOXICOLOGY WORKS
example of a teletoxicology information system; deployed in
2013, it is currently in use by 11 PCCs from 9 States in Brazil,
Tele- Poisoning Information Computational
Reference and is available 24/7.
consultation Control Database Toxicology
[22] X
Since its adoption, DATATOX has managed over 81,000
[23] X occurrences of poisoning caused by drugs, pesticides (both do-
[21] X mestic and industrial), rat poison, household cleaning products,
[24] X
[25] X
cosmetics and toiletries, industrial products, metals, abuse
[26] X drugs, plants and fungi, food, venomous, and poisonous anim-
[27] X als. Medical doctors are the most common requesters, followed
[28] X X
[29] X
by relatives and nurses. Undergraduate medical students, in
[30] X turn, coordinated by a medical doctor, are the most common
[31] X attendants.
DATATOX offers 3 main functionalities to its users: case
registry, case review, and case follow-up. The case registry
questers provide information about the poisoning occurrence. is used by attendants to register a poisoning or informative
A requester can be a patient, a person responsible for a patient, case, that may be corrected or improved by a physician using
a health care professional looking for educative material, or the case review functionality. The case follow-up allows the
a common person looking for information. Attendants are re- attendants to further improve the information about the case,
sponsible for the record of information provided by requesters, getting in touch with requesters if needed. The DATATOX in-
evaluating the severity of the reported case and eventually formation system does not have a toxicology database module,
conducting requesters to a course of action. Physicians, in turn, so on-duty attendants or physicians rely on external databases
advise and supervise groups of attendants, being responsible such as TOXBASE[36] c and Micromedex Solutions[37].
c
for the management and organization of toxicology database A common scenario involving DATATOX is as follows: the
resources. PCC receives a telephone call made by a requester. The call
Fig. 1 depicts the workflow of a teletoxicology information is answered by an attendant, who registers all information
system, including its stakeholders. regarding the poisoning case within the information system
As can be seen in Fig. 1, the requester contacts the PCC and proceeds to assess the current situation of the patient. In
providing all information available for the case (1) (e.g. patient order to support its assessment, the attendant might search
age, patient weight, intoxication pathway). In cases involving external toxicology databases or ask directly to a supervising
drugs, it is important to report what kind of drug was used physician for information. After its assessment, the attendant
and/or the quantity consumed. The attendant interacts with the gives the proper feedback to the requester, and the call is
poison control information system (2), assessing the severity ended. If necessary, the attendant schedules a follow-up in the
of the case. If necessary, the attendant can search for comple- DATATOX information system with the requester, that might
mentary information in toxicology databases (either in hard occur when a laboratory analysis arrives, or just to check the
copies or in electronic form) (3.a); alternatively, the attendant patient’s evolution. The case is closed only after a careful
may ask for the opinion of a physician (3.b). The physician, in analysis performed by a physician.
turn, may also consult toxicology databases giving feedback
B. Software quality
(5) to the attendant. Finally, the attendant gives feedback to
the requester (6) with the appropriate recommendations (e.g. Software quality is defined as the degree to which a system
first aid to be applied, medication to be prescribed). satisfies the stated and implied needs of its various stakehold-
ers [14]. Being an abstract concept, it is difficult to measure or
evaluate software quality [38]; therefore, quality models and
standards are proposed defining a set of characteristics and
relationships between characteristics as a basis for specifying
quality requirements and quality evaluation [39].
Several quality models were proposed since 1970, which
recently have been integrated into the industry quality standard
ISO/IEC 25010. This standard is divided into two models:
the quality in use model that defines five quality character-
istics related to the user interaction (effectiveness, efficiency,
satisfaction, freedom from risk, and context coverage) and
the product quality model, which defines eight characteristics
related to static software properties (functional suitability,
performance efficiency, compatibility, usability, reliability, se-
Figure 1. A common workflow supported by teletoxicology information
systems. curity, maintainability, and portability). These characteristics

32
are then decomposed into subcharacteristics, and finally into
measures. In order to be widely adopted, the ISO/IEC 25010
standard is defined in a generic manner applicable to any kind
of software system; however, it must be customized to the
evaluation of specific types of software systems in order to
ensure that particular product functionalities meet the user
needs [15].
1) The AdEQUATE Model: AdEQUATE is a quality model
developed based on the ISO/IEC 25010 standard, whose
objective is to evaluate the software quality of telemedicine
and telehealth systems from the perspective of its end users
[16]. The model systematically decomposes the quality charac-
teristics into measures, and pre-defines a questionnaire to op-
erationalize data collection. The questionnaire13 is composed
of 70 items:
• 2 demographic items, discriminating the role of the
respondents within the system and its experience in using
the system;
• 68 systematically derived items from the quality char-
acteristics and subcharacteristics found in the referred
standard.
The response format for the questionnaire items is a 4-
point Likert scale, including three additional options: “not
applicable”, “do not know”, and “did not understand the item”.
All items are accompanied by counterexamples, formulated as
positive assertions.
III. M ETHODOLOGY
In order to assess the software quality of the DATATOX
information system in the context of ABRACIT’s poison
control centres, we defined and executed a case study. This
Section characterizes the case study, and section IV presents
the analysis of the acquired data.
A. Case study definition
Based on our SLR and experience with the DATATOX infor-
mation system, we considered all software quality characterist-
ics and subcharacteristics proposed by the AdEQUATE model
as relevant13 . In order to operationalize data collection, we
revised the questionnaire items and exemplified each item with
a counterexample extracted from the context of teletoxicology
systems and the routine of poison control centres.
B. Case study execution
From November, 2015 to December, 2015 we collected
data using on-line questionnaires available via LimeSurvey14
answered by DATATOX users from multiple work shifts in all
available facilities. In the main facility users were personally
invited to answer the questionnaire; in other facilities, users
were invited by telephone. A total of 49 users were invited
and answered the questionnaire. From respondents, 65% were
attendants and 35% were physicians. Also, 55% of them used
the system for more than a year, 31% used the system for
13 The full range of AdEQUATE characteristics, subcharacteristics and
questionnaire items can be accessed in https://arquivos.ufsc.br/f/61d4095f6a/
33
more than six months but less than a year, and only 14% of
respondents used the system for less than six months.
IV. DATA A NALYSIS AND I NTERPRETATION
Data collected during the case study execution are analysed
using two metrics, applied to every AdEQUATE characteristic
computing the distribution of each response option available
to respondents.
Metric #1: For every questionnaire item, the percentage of the
sum of response options chosen by respondents.
The chart in Fig. 2 shows the distribution of each response
option grouped by ISO/IEC 25010 quality characteristics
within AdEQUATE. It is observed that for the majority of
characteristics, the answers “totally agree” or “agree” (light
green and dark green bars) correspond to more than 50% of
the responses; exceptions are “compatibility”, “reliability”, and
“context coverage”. Since the “don’t understand” option was
chosen in a maximum of 2% of responses, it is considered that
all characteristics were well understood by the respondents.
To further comprehend the quality degree of each character-
istic, we calculated a score value according to (1). In the equa-
tion, R̃i corresponds to the median of responses given on the
questionnaire for the subcharacteristic n by respondent i, and
W (R̃i ) corresponds to a weight function based on a response
option assuming the following values: 3 for “totally agree”, 2
for “agree”, 1 for “disagree”, and 0 for “totally disagree”. The
response options “don’t know”, “non applicable”, and “don’t
understand the item” are ignored. The last component in (1)
guarantees that Sn varies from 0 (poorest degree of quality
– i.e. all 49 respondents choose “totally disagree”) to 100
(exceptional degree of quality – i.e. all 49 respondents choose
“totally agree”).
 
49
100
Sn = ∑ W (R̃i ) ∗ (1)
i=1 147
Using (1), the second metric is defined as follows:
Metric #2: The median of the scores for every questionnaire
item and characteristic.
We adopted the scoring system defined by the System
Usability Scale (SUS) [40], because it is related to software
quality and it is a well established evaluation tool in the
14 https://www.limesurvey.org/
Figure 2. Distribution of questionnaire answers between characteristics.
industry [41]. It considers four response options (“Totally
agree”, “agree”, “disagree”, and “totally disagree”), resulting
in the chart depicted in Fig. 3.
Differences between Fig. 2 and Fig. 3 show the influence
of weighting a response option in the final result. While Fig.
2 shows “efficiency” and “effectiveness” roughly as having
the same quality degree, Fig. 3 shows that “efficiency” has
been better evaluated because it had more “totally agreed”
responses. This difference is more evident in the case of
“reliability” and “compatibility”. “Compatibility” had more
“totally agree” responses, but also had more “totally disagree”
responses; the result is that “reliability” could be perceived
with a better quality degree than “compatibility”. On the other
hand, other characteristics were not affected by the change
of metrics: “Satisfaction” and “context coverage” are still
perceived as having, respectively, the highest and the lowest
quality degree. Finally, since every characteristic had a score
that was over two interquartile ranges (IQR) away from the
median, we did not identify outliers for the metric #2.
V. D ISCUSSION
Data acquired during the case study execution attest an
overall good quality for the DATATOX information system.
The division of Fig. 3 into quartiles shows how end users
perceive the system related to its quality characteristics: “Sat-
isfaction”, “efficiency”, and “effectiveness” have the highest
quality degrees, followed by “risk mitigation”, “security”, and
“functional suitability”; “performance efficiency” and “usabil-
ity” were perceived as having a slightly lower quality degree,
and “reliability”, “compatibility”, and “context coverage” were
perceived as having a low quality degree. The low quality
degree perceived for “context coverage” is due to a low degree
of flexibility: The system can be used only on Mozilla Firefox,
only on desktop and laptop computers, and cannot be accessed
by requesters. Furthermore, end users consider the system as
being not optimal to be used in the toxicology context.
DATATOX is a web-based teletoxicology system, being
susceptible to fluctuations in the quality of the Internet con-
nection. Such instabilities contribute to a low degree of “fault
Figure 3. Scores computed for each characteristic.
34
tolerance”, perceived by analysing data from the case study.
DATATOX, also, does not have any integrated toxicology
database, forcing its users to rely on external data sources
and contributing to its low degree of “interoperability”.
A. Threats to validity
To enhance the internal validity and avoid selection bias in
our case study we invited DATATOX users from all available
facilities, from multiple working shifts, personally or via
telephone calls. To minimize bias in the research design,
we adopted the AdEQUATE model. Although the model has
not yet been validated by a larger study, we assume that its
systematic derivation from a largely accepted international
quality standard – the ISO/IEC 25010 standard – reduces
construction validity threats.
Due to our focus be on a specific teletoxicology system –
the DATATOX system – results of this study have a limited
external validity. Also, although it is not common to evaluate
quality through the use of questionnaires, the annex C in
the ISO/IEC 25010 standard states that all subcharacteristics
can be evaluated by external measures – degree to which a
software product enables the behaviour of a system to satisfy
stated and implied needs for the system, including the software
to be used under specified conditions [14]. Furthermore, the
ISO/IEC 25022 standard proposes the use of questionnaires as
a method to assess quality [42].
VI. C ONCLUSION
Poison control information systems help toxicologists to
manage poisoning cases, handling data regarding the poi-
soning agent, the victim, and controlling the follow-up for
the cases as needed. Some systems can help toxicologists
even further by offering a specialized toxicology database. In
this context, software quality is an issue because there is no
specific standard guiding the development of such systems,
neither a software quality evaluation model. While ISO/IEC
25010 presents characteristics and subcharacteristics allowing
software engineers to evaluate general software quality, the
standard misses the specificities of the health care domain.
In this work we analysed DATATOX, a teletoxicology infor-
mation system used by a number of PCCs in Brazil, through
a case study based on the AdEQUATE evaluation model. The
performed analysis shows that the DATATOX system is per-
ceived as having a good overall quality degree; its strong points
are related to satisfaction, efficiency and effectiveness, reveal-
ing a system that satisfies users and allows them to execute
tasks with accuracy and completeness, spending few resources.
On the other hand, DATATOX was considered to have low
reliability, compatibility and context coverage, meaning that
the system does not always perform some functions, does not
exchange information with other systems, and is not flexible
enough to cover all contexts of use. Additionally, this work
shows the feasibility on evaluating software quality from the
perspective of end users of poison control information systems
using AdEQUATE’s systematically derived questionnaires.
 Future works in this topic could focus on the validation [20] J. Kern, “Evaluation of teleconsultation systems,” International journal
of medical informatics, vol. 75, no. 3, pp. 330–334, 2006.
of the AdEQUATE model by evaluating other poison control [21] S. Grossman and T. Zerilli, “Health and medication information re-
information systems, providing better insights on their per- sources on the world wide web,” Journal of pharmacy practice, pp.
ceived quality, and also evaluate other teletoxicology systems 85–94, 2013.
[22] P. Wexler, S. G. Gilbert, N. Thorp, E. Faustman, and D. D. Breskin, “The
in contexts other than public health care (in which DATATOX world library of toxicology, chemical safety, and environmental health
is inserted). (wlt),” Human & experimental toxicology, vol. 31, no. 3, pp. 207–214,
2012.
ACKNOWLEDGMENT [23] M. F. Cesta et al., “The national toxicology program web-based non-
neoplastic lesion atlas a global toxicology and pathology resource,”
This work was supported by CNPq (Conselho Nacional Toxicologic pathology, vol. 42, no. 2, pp. 458–460, 2014.
de Desenvolvimento Cientı́fico e Tecnológico) and FAPESC [24] A. de Souza et al., “An overview of the challenges in designing,
(Fundação de Amparo à Pesquisa e Inovação do Estado de integrating, and delivering bard a public chemical-biology resource
and query portal for multiple organizations, locations, and disciplines,”
Santa Catarina). Journal of biomolecular screening, pp. 614–627, 2014.
[25] M. R. Cummins, B. Crouch, P. Gesteland et al., “Building a knowledge
R EFERENCES base for health information exchange between emergency departments
and poison control centers,” in System Science (HICSS), 2012 45th
[1] L. Nelson et al., Goldfrank’s Toxicologic Emergencies. McGraw-Hill
Hawaii International Conference on. IEEE, 2012, pp. 910–914.
Medical New York, 2011.
[26] M. Cases et al., “The etox data-sharing project to advance in silico drug-
[2] L. Perry et al., “National toxicovigilance for pesticide exposures result-
induced toxicity prediction,” International journal of molecular sciences,
ing in health care contact–an example from the uk’s national poisons
vol. 15, no. 11, pp. 21 136–21 154, 2014.
information service,” Clinical toxicology, vol. 52, no. 5, pp. 549–555,
[27] C. Hochstein, D. Gemoets, and J. Goshorn, “Ten years of change:
2014.
National library of medicine toxmap gets a new look,” Medical reference
[3] Centro de Informacões Toxicológicas de Santa Catarina, “Estatı́sticas
services quarterly, vol. 33, no. 4, pp. 428–437, 2014.
anuais,” 2015, date last accessed 2015-12-11. [Online]. Available:
[28] J. K. Maddry et al., “Wartime toxicology: evaluation of a military
http://www.cit.sc.gov.br/site/?page id=464
medical toxicology telemedicine consults service to assist physicians
[4] Centro de Informacões Toxicológicas do Rio Grande do Sul,
serving overseas and in combat (2005–2012),” Journal of medical
“Estatı́sticas dinâmicas,” 2015, date last accessed 2015-12-11.
toxicology, vol. 10, no. 3, pp. 261–265, 2014.
[Online]. Available: http://www.cit.rs.gov.br/index.php?option=com -
[29] K. Kongsbak, N. Hadrup, K. Audouze, and A. M. Vinggaard, “Applicab-
wrapper &view=wrapper&Itemid=60
ility of computational systems biology in toxicology,” Basic & clinical
[5] World Health Organization et al., “Guidelines for poison control,” 1997.
pharmacology & toxicology, vol. 115, no. 1, pp. 45–49, 2014.
[6] J. Mowry et al., “2014 annual report of the american association of
[30] P. R. Chai, K. M. Babu, and E. W. Boyer, “The feasibility and
poison control centers’ national poison data system (npds): 32nd annual
acceptability of google glass for teletoxicology consults,” Journal of
report,” Clinical toxicology, vol. 53, no. 10, 2015.
Medical Toxicology, vol. 11, no. 3, pp. 283–287, 2015.
[7] W. Tormey and T. Moore, “Poisonings and clinical toxicology: a
[31] P. R. Chai et al., “The virtual toxicology service: Wearable head-
template for ireland,” Irish journal of medical science, vol. 182, no. 1,
mounted devices for medical toxicology,” Journal of Medical Toxico-
pp. 17–23, 2013.
logy, vol. 10, no. 4, pp. 382–387, 2014.
[8] Poison Control Centers, 2015, date last accessed 2015-12-14. [Online].
[32] American Association of Poison Control Centers, “Npds elements,”
Available: http://triage.webpoisoncontrol.org/
2015, date last accessed 2015-12-14. [Online]. Available:
[9] A. Skolnik, “Telemedicine and toxicology: back to the future?” Journal
http://www.aapcc.org/data-system/npds-elements/
of Medical Toxicology, vol. 9, no. 3, pp. 217–219, 2013.
[33] New Zeland National Poison Center, “Hints and tips for calling
[10] D. N. Bateman, A. M. Good, W. Laing, and C. Kelly, “Toxbase: poisons
a poison centre,” 2015, date last accessed 2015-12-14. [Online].
information on the internet,” Emergency medicine journal, vol. 19, no. 1,
Available: http://www.poisons.co.nz/calling.php
pp. 31–34, 2002.
[34] California Poison Center, “About the california poison control
[11] S. M. Seifert et al., “An analysis of energy-drink toxicity in the national
system,” 2015, date last accessed 2015-12-14. [Online]. Available:
poison data system,” Clinical toxicology, vol. 51, no. 7, pp. 566–574,
http://www.calpoison.org/about/home.html
2013.
[35] Maryland Poison Center, 2015, date last accessed 2015-12-14. [Online].
[12] F. Gough et al., “Ata practice guidelines for live, on-demand primary
Available: http://www.mdpoison.com/
and urgent care,” Telemedicine and e-Health, vol. 21, no. 3, pp. 233–241,
[36] Toxbase, “The primary clinical toxicology database of the national
2015.
poisons information service,” 2015, date last accessed 2015-12-14.
[13] S. Wagner, “Quality models,” in Software Product Quality Control.
[Online]. Available: https://www.toxbase.org/
Springer Berlin Heidelberg, 2013, pp. 29–89.
[37] Micromedex, “Web application access,” 2015, date
[14] International Standard Organization, “Iso/iec 25010: Systems and soft-
last accessed 2015-12-14. [Online]. Available: ht-
ware engineering – systems and software quality requirements and
tps://www.micromedexsolutions.com/home/dispatch/ssl/true
evaluation (square) – system and software quality models,” Tech. Rep.,
[38] A. Gillies, Software quality: theory and management. Lulu. com, 2011.
2014.
[39] International Standard Organization, “Iso/iec 9126-1:2000, software
[15] I. Biscoglio and E. Marchetti, “An experiment of software quality
engineering – product quality - part 1: Quality model,” Tech. Rep., 2000.
evaluation in the audio-visual media preservation context,” in Quality
[40] J. Brooke, “Sus-a quick and dirty usability scale,” Usability evaluation
of Information and Communications Technology (QUATIC), 2014 9th
in industry, vol. 189, no. 194, pp. 4–7, 1996.
International Conference on the. IEEE, 2014, pp. 118–123.
[41] ——, “Sus a retrospective,” Journal of Usability Studies, vol. 8, no. 2,
[16] J. M. Alves, C. Wangenheim, T. Lacerda, A. Savaris, and A. Wan-
pp. 29–40, 2013.
genheim, “Adequate software quality evaluation model v1.0,” Instituto
[42] International Standard Organization, “ISO/IEC 9126-2:2003 Software
Nacional para Convergência Digital – INCoD, Tech. Rep., 2015.
engineering – Product quality – Part 2: External metrics,” Tech. Rep.,
[17] S. H. Thomas, “An agenda for uk clinical pharmacology: Developing
2003.
and delivering clinical toxicology in the uk national health service,”
British journal of clinical pharmacology, vol. 73, no. 6, pp. 878–883,
2012.
[18] A. Namera et al., “Simple segmental hair analysis for α-
pyrrolidinophenone-type designer drugs by monospin extraction for
evaluation of abuse history,” Journal of Chromatography B, vol. 942,
pp. 15–20, 2013.
[19] B. Kitchenham, “Procedures for performing systematic reviews,” Keele,
UK, Keele University, vol. 33, no. 2004, pp. 1–26, 2004.

35