Trust in Health Websites: A Review of an Emerging Field

Laurian Vega
Computer Science 2202 Kraft Drive Virginia Tech, Blacksburg, VA

Enid Montague, Ph.D.
Industrial and Systems Engineering 3107 Mechanical Engineering Building University of Wisconsin-Madison, WI

Tom DeHart
Computer Science 322 BSB, 227 Penn Street Rutgers University, Camden, NJ

Laurian@vt.edu

EMontague@wisc.edu

tdehart@vt.edu

ABSTRACT
As people increasingly turn to health websites for the purposes of self-diagnosis and healthier living, we have an obligation to evaluate the factors that might affect a given user's assessment and their willingness to use such sites. Constructs such as quality, trust, and credibility need to be defined within this space in order for us to truly understand how and why people use health websites. In an effort to better understand these constructs we conducted a comprehensive analysis of all peer-reviewed empirical studies on trust in health websites -- this paper is the result. Work on this topic was provided from eleven fields including HCI, Informatics, Medicine, and Decision Making. Our findings show that authors often value different facets of trust, report different outcomes, and rarely cite each other. Without a coherence of terms and values, the task of presenting and understanding how users trust health information on the web will be intractable.

has, however, impacted the traditional health care relationships. The Pew Internet Foundation recently reported that 61% of Americans are currently using the internet for health information, and 20% use the internet to look up information about medicine and experimental treatments [15]. This rise in use of health websites indicates a need to understand how users evaluate the credibility of online information in order to better understand the impact it will have on the patient’s trust in different parts of the health system. While there has been prior work modeling user trust of various technologies [18, 30], research has demonstrated that trust in one system or artifact does not necessarily translate to another system or artifact [43]. For example, while there may be valid models of trust that workers place in factory machinery, those same models of user trust may not apply to eCommerce websites. User trust models should then be created and evaluated specific to the domain of application in order to understand the nuances of each particular system. Likewise, this practice should be used even when attempting to understand different models of trust in online activities (i.e. looking up medical information v. purchasing a book). For this reason, current models of how a patient trusts their health care provider cannot be merged with current models of how users trust eCommerce websites. Instead, there exists a need to understand what antecedents induce or affect trust, while also understanding the facets and components that make up the construct of a user’s trust in health websites. In this paper we evaluate the body of empirical research on user trust in e-health websites. This research is comprised of 49 papers collected in Summer 2009 stemming from a multitude of different backgrounds such as HCI, Medicine, and Informatics, and Decision Making. The data and analysis used in this study was created in conjunction with a corresponding study that examined differences in trust measurements and varying construct definitions as a function of time. It has been submitted as a separate journal publication. For this paper we reviewed 49 publications in the realm of trust with one particular question in mind -- one that is of particular important to the IHI audience: how does research from different disciplines impact the comprehensive understanding of trust in health websites? Our goal is to provide an example evaluation method for understanding emerging multidisciplinary constructs, like trust and credibility, for web communities. This framework will provide a method to analyze the contributions and parlance of research on social constructs. To meet this goal we 1) used content analysis to organize constructs presented in the manuscripts 2) conducted a social network analysis, and 3) linked

Categories and Subject Descriptors
K.4.3 [Computers and Society]: Organizational Impacts – Computer supported collaborative work; H.1.2 [Information Systems]: Human Factors.

General Terms
Reliability, Human Factors, and Theory.

Keywords
Health, Trust, Credibility, Review, Empirical Evaluation, Human Factors, Human-Computer Interaction, Websites

1. Introduction
Trust, or the lack there of, is a foundational part of any relationship. In particular it is critical to relationships in the realm of healthcare. For instance, the patient trusts that their team of healthcare professionals is going to manage their wellness appropriately. Health care professionals additionally trust each other to work cooperatively and to effectively communicate pertinent patient information. The increase in use of the internet
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. IHI’10, November 11–12, 2010, Arlington, Virginia, USA. Copyright 2010 ACM 978-1-4503-0030-8/10/11...$10.00.

facets to empirical findings from each discipline. We address three key concerns: that the construct of trust is overly fragmented by domain; that researchers on trust in health websites lack a clear definition and what definitions they have are conflated; and, that there is distinct lack of collaboration between authors. We conclude with a discussion about the future of research on trust within the larger socio-technical system and an analysis of the user’s assessment of a website’s credibility.

important quality outcomes such as adherence, satisfaction, and improved health outcomes [1]. Models of trust provided by related disciplines, such as eCommerce, are valuable for highlighting domains that might be correlated, but cannot account for intrinsically different relationships and factors that will impact trust in health websites. It is for this reason that research trust in health websites is critical in a field that has an increasing impact on daily health related interactions.

2. Background & Related Work
Prior work has demonstrated an increase in the use of health technology [37, 61, 71]. A major factor in this increase is the push to move to a paperless medical work environment. And, in doing so, there is an assumption that there will be an increase in the availability of credible health information online [29]. However, the user’s employment of health websites can vary from usergenerated content, such as Wikipedia, to more authoritative websites, such as WebMD or IHI.gov. Understanding the impact that these web technologies have on the patient’s perceptions of quality of care and on the patient-provider relationship can impact the design, implementation, and integration of health websites into the larger health care system and the adoption of electronic medical records. A health website, also known as a health infomediary, is a company that supplies health information: “an online provider that offers information, advice, guidance, and assessment for wellness and health-related issues, including referrals and wellness-related evaluations” [74]. Sillence et al. have also identified different types of web technology that can be labeled as health websites [60]. These include web portal sites, support groups, charity sites, government sites, pharmaceutical sites, sales sites, personal sites, medical databases, media sites, and clinician sites. We have adopted Sillence et al.’s definition of an e-health website as the focus for our study. Research on user mental models of trust in technology has been reviewed from multiple relevant areas [18, 44, 46]. For example, the work on user trust in automation examined how the design of technology impacts trust in the automation technology [46]. While this work demonstrates how users evaluate automated technologies, it does not take into account the particularities of trust in a website and the impact that trust has on related relationships. In particular, factors such as ‘consistency’ that are paramount to user trust in automation do not necessarily translate into the realm of trust in health websites where there may be dynamic content. Additionally, in the related work of trust in website, few models account for interpersonal relationships that are impacted by constructs like trust or similar constructs like credibility, authority, and expertise within relationships [52] such as the doctor-patient relationship. A second group of related research explores user trust in webbased technologies, such as eCommerce websites [32, 38, 41]. Research in this area examines the relationship between user trust and the resulting behavior (e.g., purchasing, supplying personal information) [18]. While this research is similar, it does not account for the impact that the resulting behavior then has on external relationships – such as the patient-provider relationship. We focus on this relationship in particular given that of people who access health information online, 50% of them report that they then talk to their doctor about the found information [72]. Additionally, the trust in the health provider has been linked to

2.1 Trust and Credibility
Conceptualization of the construct of trust customarily will focus on defining trust’s facets and antecedents. Facets of trust are the constructs that it may encompass or make up user trust. Examples taken from the body of reviewed literature include 'wellintentioned' [67], 'truthfulness' [52], and 'integrity' [74]. Antecedents of trust, on the other hand, are the external factors that impact or cause variation in a user’s trust. Examples include 'relevance', 'authority', or 'disposition' [30]. In the relevant literature, the constructs of credibility and trust are linked through their facets and antecedents. In forty-nine papers, only thirteen included definitions of trust. Of these thirteen papers, five papers defined trust as a facet of the construct of credibility [14, 23, 52, 67, 72]. Heavily citing the work of Hovland et al. [24], trust was defined as the well-intentioned, unbiased, and truthful nature of the source of the information. Along with expertise, which is defined as the quality of the source of information, trust is used to understand the credibility of a source of information. This conceptualization of the relationship of trust and credibility, however, is only one variation. In our conception of trust in health websites we argue that credibility is a part of trust. In a sociotechnical system multiple humans work with technologies and are influenced by aspects external to the system as well [49]. Interactions with and surrounding health websites are more than individual patients working with an interface: they are portions of a complex health care work system. Patients have primary relationships with their care providers and those that provide care for them. Trust is therefore constructed at different levels of that system; patients can have trust relationships with care providers [1, 68], health institutions [21], and health technologies [44]. A person’s trust in a health website is formed on the system surrounding the website and the health system that the patient is a part of. For example if a patient finds incorrect information on an electronic medical record, is their trust diminished in the website that hosts the record, the company that provides the record transferring, or the health institution or care provider? Credibility would be placed in the company that designs the health record system, but it is trust that will influence the patient to either use the system or like systems again and perhaps other aspects of the system. Given this understanding, we present our work that reviews user evaluations of health websites.

3. Method
Three methods of analysis were used in order to understand how trust might impact the design and implementation of health websites and, more importantly, how it may affect the use of such sites. After narrowing down the body of work to 49 papers, we conducted a word frequency analysis, a social network analysis, and, finally, a unique analysis of empirical outcomes by discipline; all of which are explained below. While methods like

Figure 1. The pie graphs depict the use of the factors 'quality' and 'behavior in reference to the research areas that use them. For example, 30% of the uses of the word 'quality' came from the area of Medicine. these may lack statistical significance in such small data sets, the results can still prove insightful as is shown below. meta-constructs (e.g., ‘difficult’ and ‘difficulty’ were combined). Stop words such as 'available', 'care', 'independent', 'positive', 'positively', 'control', and 'certain’ were omitted as these terms were often used outside of the context of explaining trust (e.g., “Independent sample t-tests were employed…”). Scripts were then used to count the terms across all 49 papers, with each term being counted a maximum of one time per paper. Our goal with this analysis was to gain a richer understanding of what the research community believes to be related to trust in health websites. To do this we had to go beyond provided definitions, facets, and antecedence, and examine the words that people used through discussing the construct of trust. It was our hypothesis that research from different domains would utilize different constructs related to trust and with different frequencies.

3.1 Data Sources
A comprehensive search of thirteen databases (e.g., ScienceDirect) was completed in the Summer 2009 using search terms such as *trust*, website*, webpage*, online*, internet*, web site*, web page*, health*, and medic*. The initial search produced over 2,000 articles, which were then evaluated individually by two reviewers based on pre-established criteria of being peer-reviewed, empirical, and including trust as a variable. Disagreements were discussed and then agreed upon for the final set. Supplementary computer searches were conducted from articles that appeared in the reference sections of the set of articles and identified as related; 49 relevant articles were identified and examined by the authors for the purposes of this paper. Included papers represented both experimental and theoretical peerreviewed conference proceedings and journal papers. Excluded from the set were publications unrelated to the topic, those not peer-reviewed, and those yet to be published.

3.2.2 Social Network Analysis
Social network analysis maps and measures relationships between entities; in this study, social network analysis was used to analyze ideas about trust in health websites. Our social network analysis explored articles and looked for citation patterns between authors to explore which entities were citing other. The outcome was used to generate hypotheses about the flow of ideas and shared conceptualizations of the definition of trust. The nodes in our network are the authors of the papers while the links show relationships or flows between the nodes. Social Network analysis provides both a visual and a mathematical analysis of relationships. More importantly social network analysis can help visualize shared and disparate ideas in a discipline. Several analyses were conducted using NodeXL social network analysis software: 1. Degree: The number of vertices that the node is connected to. In this study, Degree is the number of author articles that the paper is connected to. 2. Betweenness centrality: The extent that a node acts as a connection to other nodes. In our review this is operationalized to mean that a paper may function as an interconnection between different areas of trust research.

3.2 Analysis Method
We employed a meta-analysis framework to examine the impact of a research discipline on the concept of trust. This method is appropriate given that a review of trust in health is novel and this method has been used for similar work (see [12] for example). Three strategies were used to analyze the papers for the impact of research on trust by discipline: (1) content analysis to examine the frequency of words used to discuss trust research, (2) social network analysis to examine how researchers from different areas cited each other, and (3) an analysis of empirically found research outcomes by research area.

3.2.1 Content Analysis of Word Frequency
To understand how researchers from different backgrounds would discuss and evaluate the construct of trust we analyzed the content of each paper for the frequency of terms (see [10] for an example). A list of three hundred and ninety-six terms was compiled from a thesaurus and from articles [28, 45] that employed empirical methods to derive terms related to user trust and technology. These terms were then combined to form 108

3. Closeness centrality: The measure of the average shortest distance from each node. A lower centrality score indicates a more important or central position in the network. 4. Eigenvector centrality: an adjusted measure that reports a combination of how many connections a vertex has with the importance (degree) of the connections. The areas of research were color coded by clusters in Figure 2. Clusters are groups of papers that are interconnected. They are determined with a cluster coefficient, which measures how connected a vertex’s neighbors are to one another (i.e. creating "cliques" of how connected authors and articles are to one another based on citation patterns) [64]. Papers that did not cite any other papers in the body of reviewed work were not included in this analysis. Table 1. This table depicts the interconnected papers in a social network analysis with the associated degree, betweenness, closeness, and eigenvector centralities. Eigenvector Centrality Betweennes Centrality

3.2.3 Analysis of Outcomes by Discipline
To evaluate the outcomes from empirical research on trust in health websites a comprehensive list of all outcomes was created. These outcomes were then organized by what trust can and cannot affect, and factors that affect user trust in health websites. These outcomes were then coded according to which realm of research the paper stemmed from (e.g., the Medical field; a list is available in Figure 1). Papers could contain multiple findings that would span multiple categories. Two members of the research team negotiated and reached agreement on the outcome categorizations. Our goal with this analysis was to determine the full spectrum of empirical research outcomes that were found to affect trust in health websites. The hypothesis was that the overall research spectrum was fragmented by the area that the paper stemmed from. Specifically, outcomes were combined to create more than 60 factors that were then combined to create five categories. The five categories were: Usability, Content of the Webpage (e.g., the presence of advertisements and the presence of real life examples), Information Factors (e.g., quality, adequacy, and personalization of the information), Contextual Factors (e.g., access location and consistency with current beliefs), and User Demographics (e.g., gender, health, and age). These categories were then further decomposed into sets of empirically determined factors. For example, Gender was an empirically derived factor that was within the category of Demographics. To assess whether categories and factors were correlated with particular areas, the outcomes were then coded by discipline.

Sillence et al. (2004) Dutta-Bergman (2003) Rains (2007) Lemire et al. (2008) Rains &Karmikel (2009) Sillence, et al. (2005) Bates et al. (2006) Sillence et al. (2007a) Walther et al. (2004) Hong (2006) Huntington et al. (2004) Sillence et al. (2007c) Fisher et al. (2008) Khoo et al. (2008) Menon et al. (2002) Glenton et al. (2006) Newnham et al. (2006) Newnham et al. (2005) Sillenceet al. (2006) Sillence et al. (2007b) Zahedi& Song (2008) Song &Zahedi (2007) Halkias et al. (2008) Hesse et al. (2005) Wainstein et al. (2006) Semere et al. (2003)

6 4 4 4 4 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 1 1

0.962 0.494 0.323 1 0.105 0 0.226 0 0.005 0.005 0.947 0.068 0.902 0.015 0 0 0 0 0 0 0 0 0 0 0 0

3.063 2.875 2.938 2.5 3.438 3.938 3.688 3.938 3.688 3.688 2.563 3.188 2.75 1 4.625 4 1 1 4 4 1 1 1 1 1.5 1.5

Closeness Centrality

Degree

Vertex

0.054 0.45 0.445 0.416 0.395 0.026 0.159 0.026 0.267 0.267 0.153 0.258 0.066 0 0.051 0.017 0 0 0.017 0.017 0 0 0 0 0 0

3.3 Data
All papers included in our review were published between 2001 and 2009. 2001 is the earliest date that papers on this topic were published. The papers were organized into background categories. These categories were based on the journal or conference in which each paper was published (e.g., a paper published at an eCommerce conference would be put into the eCommerce area). The areas are as follows; communication (n=3) [25, 35, 51]; decision making (n=2) [53, 65]; human-computer interaction, computer science, and the internet –called Computing (n=9) [11, 13, 52, 58, 59, 60, 61, 62, 67]; health and medical informatics and health – called Health Informatics (n=11) [3, 9, 16, 20, 27, 33, 37, 54, 72]; health services and healthcare (n=3) [6, 8, 17]; information and library science (n=3) [23, 26, 39]; media and society (n=1) [14]; medicine (n=14) [2, 3, 5, 7, 19, 22, 31, 34, 42, 47, 48, 56, 66, 71]; pharmacology (n=1) [69]; and, social sciences, psychology, and sociology – called Social Sciences (n=3) [4, 50, 63]. Rosenbaum et al. paper is counted twice because it is published in BMC Medical Informatics and Decision Making, which spans two areas.

3.4 Content Analysis of Word Frequency
The top ten words used in discussing the research on trust in health websites across all reviewed papers were: quality (47), understanding (44), reliability (43), communication (41), experience (40), knowledge (39), accuracy (39), credibility (38), concern (38), and behavior (38). The top fifty most frequently used words were found in 37% of all the papers.

To analyze the parlance of trust by field of study, we charted the frequency of terms that were present in each discipline. With three areas encompassing 69% of the total group of papers, these groups had a large impact on the frequency of use. For example, the construct of ‘quality’ was used by every paper in every group except for the groups of Computing and Health Services & Healthcare. ‘Quality’ is used 70% of the time in the papers with the largest base of work. (Figure 1 depicts the factors of quality and behavior by area and percentage of use.) In contrast to ‘quality’, the construct of ‘behavior’ is still used 56% of the time by the three areas with the largest base of papers. This is a 14% decrease in use by the three largest groups. A pattern in use and weight can be detected. The areas of Medicine, Health Informatics, and Computing for most constructs make up the largest portion of their use. This is to be expected given that they have the largest base of papers. However, as the amount of use decreases among the three largest groups, the remaining areas use of those words remains relatively high. This can be seen in use the construct of ‘behavior’ with 31% of the papers making up 44% of the usage. Other examples include the term ‘negatively,’ which is used in all Communication and Decision Making papers; whereas the term ‘negatively’ is only used 44% of the time in the computing research and 29% of the time in the research on Medicine.

continues.

3.6 Analysis of Outcomes by Area
To analyze the outcomes of trust research in health websites, the outcomes were sorted into sets. The first set of empirical outcomes evaluated how trust in one source would then affect trust in another. Within these outcomes, the type of health website being assessed varied by the area of research. For example, research from the area of communication focused on how trust in different sources, like the internet, physicians, and entertainment would affect how users then trusted internet information [35, 51]. Alternatively, research from the area of decision making focused on how trust in the internet as a whole would enable or disable people to look up more information about their medical condition [53, 65]. Another example is from the area of Medicine. Medical research focused on how trust in medical related online information, such as prescription drugs information, online health records, or general medical websites, all affected trust in other socio-technical systems such as all online prescription information, the national healthcare service, and the intention to use health websites, respectively [2, 19, 42]. Twelve of the reviewed papers mentioned that the internet has an effect on doctor-patient relationships. Twelve more cited this relationship as an important factor for why trust in health websites should be studied. Twenty-five papers did not cite this

3.5 Social Network Analysis
Twenty-six papers cited at least one other paper within the body of reviewed literature. Twentythree, or 47% of, papers did not include citations to other work in the body of reviewed literature instead opting to cite only within their own fields of research. Social network analysis provided insight in to the connections between articles and authors. See Table 1 for values and Figure 2 for a visual depiction. 1. Degree: The paper published by Sillence et al. in 2004 [58] had six vertices from the body of reviewed papers, which was the highest number. Followed by Dutta-Bergman [9], Lemire et al. [37], Rains [51], and Rains and Karmikel [52]. All had four degrees. 2. Betweenness centrality: Several papers did not act as a connection to other nodes, while the paper by Lemire et al. [37] reported the highest betweenness centrality score. 3. Closeness centrality: This measurement conveyed the average shortest distance from each node; lower centrality scores indicate more important positions in the network. The lower centrality measures appear towards the bottom of Table 1, indicating that there exists a disconnect between the papers that have more important positions and the papers that are highly cited, which are the top of the table. 4. Eigenvecter centrality: This measurement is an adjusted measure that reports a combination of how many connections a vertex has with the importance (degree) of the connections. After adjustments with the eigenvector, the same trend

Figure 2. This network is a representation of the social network analysis of all empirical papers on trust in health websites. Boxes represent papers and edges represent citations. Colors and location indicate clusters and connectedness. Note that colors used in this figure are not related to those used in Figure 1. Colors are used in this figure to clarify different cliques (e.g., Bates [3], Menon [42], and Dutta-Bergman [9] form a clique as these three papers cite each other).

relationship in their analyses. The second set of empirical outcomes was organized into factors and categories of outcomes that affected trust in health websites (Table 2). Given that papers could have outcomes spanning multiple categories, the breakdown by the number of papers found in each category is as follows: Usability=12, Content of the Webpage=18, Informational Factors=19, Contextual Factors=16, and User Demographics=13. Several correlations were found between the area of research and the types of outcomes. In the realm of Decision Making, both papers coded under this area corresponded to the Contextual Factors of ‘Experience & Family’ and ‘Reputation’. In the areas of Health Informatics, 45% of the papers focused on the Content of the Information, and Informational Factors. In Computing, 77% of the outcomes were related to usability, with 66% presenting outcomes that related to the user’s navigation of a webpage. The outcomes from the papers within Information and Library Science were diverse having outcomes to all five categories. This finding is similar within the papers from the area of Social Science. Social Sciences presented outcomes from all of the areas except contextual factors. Within the area of Medicine, many of the papers did not present outcomes that could be coded with our framework. However, 33% of their papers presented outcomes relating to contextual factors.

research to determine the ‘quality of care,’ can overwhelmingly impact the facets of trust studied by the research community. While it may seem remiss to focus so heavily on the separate communities that research trust, the interrelation of trust research and their disciplines proves to be notably significant. For example, Fruhling's paper [16], one in the realm of health informatics, investigates the "User Interface usability factors" of eHealth technology while Flanagin's paper [14], in the area of Media & Society, instead examines end-user "perceptions of sponsor credibility." Different disciplines ask different questions thus making the analysis of the community of trust an important topic to discuss. Our second analysis of the body of research examined the social network of how the papers were interrelated. From this analysis we derived four measurements of the number of interconnections between papers, whether certain papers served as between papers in a network and how interconnected the papers were. These measurements facilitated an analysis of whether work on trust in health websites is building off of each other. Perhaps the largest and simplest finding from this analysis is how little the research in this area is citing each other. Recall that only half of the dataset was citing each other. While the 23 papers that were not cited and did not cite any other work might be used to build research in their respective areas, they are not contributing to the research discussion on trust and health websites. Trust in health websites is still but an emerging field; however, even with only 49 relevant papers in this field, the research is still highly fragmented. The work of Sillence et al. [58] had the highest number of connections, six papers, and the second highest betweenness centrality score. However, what should be noted is that the work of Sillence et al. is a body of six papers. This does not diminish the body of work that Sillence et al. have produced, but explains the large voice that these papers make up when considering the larger body of work. Table 2. Percentage of papers by area reporting outcomes by category. Demographics 20% 36% 36% 33% 33% 33% 33% 0 0 0 Content of the Webpage Informational Factors 13% 45% 66% 0 33% 66% 33% 100% 0 0

4. Discussion
Research on trust stems from a variety of disciplines: psychology [55], sociology [57], and business [40], to cite a few. While each of these fields may not discuss the impact of user trust on health websites, their definitions of user trust can still prove useful. The empirical literature on trust and health websites encompasses 49 papers from ten areas. The first analysis of this body of research analyzed the content of each paper for the frequency of used terms. This research demonstrated that the three largest areas of research, Medicine, Health Informatics, and Computing had the largest impact on the frequency of term usage. This is to be expected since they had the largest number of papers to pull from. This analysis also found that as the frequency of terms being used decreased, the percentage of usage became more equalized by areas. While the domains of Medicine, Health Informatics, and Computing had the largest proportion of papers, it is important for the other voices to not become marginalized as work in this area becomes more prevalent. The construct of trust in health websites is multi-dimensional thus the verbiage used to discuss it should be as diverse. To explain further, the analysis of terms used is important when considering the antecedents and facets of trust in health websites; the choice of some terms over others to define and describe the construct of trust, its facets, and antecedents, are equally as important as the focus of research being on trust and health. Our research indicates that the terms being used by the areas with the highest number of publications is not an inclusive parlance. Therefore, researchers should consider terms being used across all areas when defining and using the construct of trust to understand the user’s assessment of web credibility. An equally important finding from this analysis is the fact that there was not a cohesive voice in the discussion of user trust in health websites. With different research areas focusing on different terms the discussion can become fragmented by area – indicating that research from different areas cannot build off of each other and further the overall research discussion. For example, the term ‘quality,’ which is used frequently for medical

Medicine Health Informatics Computing Communication Health Services & Healthcare Information & Library Science Social Sciences Decision Making Media & Society Pharmacology

0 18% 77% 33% 0 33% 33% 50% 0 0

0 45% 66% 0 0 66% 33% 100% 100% 0

Contextual Factors 33% 18% 18% 0 33% 66% 0 100% 100% 0

Usability

The paper with the highest betweenness score was Lemire et al. [37]. However, many papers did not function as a node connecting two pieces of work. For example, in Figure 2, the work of Song and Zahedi [65, 74] stands apart from the rest of the work. The lack of papers functioning to interconnect to each other is a further indicator of the fragmentation present in the work on trust in health websites. Closeness centrality and eigenvector centrality scores show that the more important papers also have lower degrees. This could be because of disconnected groupings such as Zahedi [65, 74] and Newman [47, 48]. Future work in this area should explore citation patterns and groupings with social networking analysis in a larger group of articles. It is important to understand the connections between theoretical papers and empirical findings and for the discipline to adopt a shared or progressive definition of trust. In our third form of analysis, we examined the empirical outcomes by the areas that the papers stemmed from. To accomplish this analysis we divided the outcomes into two collections. The first collection demonstrated how trust in one would affect trust in another. For example, one set of outcomes demonstrated that trust in general media would affect trust in health websites. Research on the transitive nature of trust demonstrated that the areas of research would impact what trustees were to be studied. Given that the spectrum of health websites is only growing as technology grows, understanding the transitivity of trust in this medium might be intractable at this time. For example, there has been little research on studying the impacts on trust of examining online health information on a smart phone versus a home PC. With smart phones becoming one of the primary methods of people accessing online information across the world, it seems like this would be a fruitful avenue of research. However, it was not one that was readily visible five years ago because of the growth of this emerging technology. Our analysis demonstrates that while outcomes in on this topic will be fragmented by the area of research, this is perhaps a constructive discourse. It is in this body of research that insight into how trust in health websites can impact the doctor-patient relationship. Approximately 25% of the papers examined this relationship and found that patient trust in health websites was affecting or was affected by the doctor-patient relationship. (See the work of [19, 35, 50].) Or, contradictorily, the work of [48] found that this relationship was not impacted. What is of value to take away from this review is that the research community on trust in health websites finds this relationship critical enough to attempt to study its impact. The fact that there is still disagreement on how and why it is affected is a clear indicator of a need for future work. The second collection of outcomes was organized into categories and factors that impacted a user’s trust in a health website. The outcomes of the papers that made up categories and factors were then coded by area. The span of the number of papers by category indicates that there is not one category of outcomes that outnumbers the others. This further shows that research in the five categories of Usability, Content of a Webpage, Informational Factors, Contextual Factors, and User Demographics might be a useful framework for examining trust in health websites. When examining the outcomes by area there were some, like Social Sciences and Library and Information Science, which did report

findings that spanned many of the categories. This indicates that while the parlance may not be the same between all the areas, the outcomes that are being derived may be reproducible and valid. This is a strong and important finding in such a new area of research. However, there were some areas that reported outcomes within particular categories of research. For example, Computing was highly focused on the usability of the health webpage. While the strength of a particular research area is its focus, it is important for research to be able to build off of the findings of others. This means that the body of research from an area should examine how their focus could voice an opinion about the findings from different areas. Unsurprisingly, prior reviews of trust in other domains find that the use and definition of trust is fragmented [18, 36]. Even within the larger research area of trust in medical/health informatics, definitions are in conflict citing trust as being a behavior and a perception [70] or as an interpersonal interdependence [73]. As fields of study mature, however, and the context of how a particular construct, like trust, is being used allows the research community congregate towards a shared definition. For instance, the work of Grabner-Krauter and Kaluscha serves as a valuable example in examining the related constructs in the study of trust and ecommerce [18]. They found that for trust in ecommerce, the ‘trust’ construct is multi-layered and reflects institutional phenomena as well as personal forms of trust. They argue, as we do as well, that understanding how the construct of ‘trust’ is being used within a particular domain is paramount to understanding the outcomes and what is really being studied. It is for this reason, that understanding what research areas are contributing to the study of trust and health websites, and how they are defining their constructs is particularly important.

5. Conclusion
In this paper we have presented a review of all empirical literature on trust and health websites. We have demonstrated how the constructs of trust and credibility are related, and how the construct of trust is varied by domain. We have addressed how the current research has affected our collective understanding of trust. Our findings could indicate that the definition of trust is fragmented across the various fields, and patterns of citation between authors in this research area. By prioritizing our analysis of the literature on the areas that the research came from, our results suggest that the research community is divided. We showed that different fields of study use different language and factors to discuss the construct of trust. The social network analysis demonstrated that researchers from different fields are not citing each other furthering the evidence of a fragmented field. This fragmentation may impede efforts to define how research from different backgrounds might impact the understanding of trust in health websites. Last, our analysis of the outcomes of empirical trust research demonstrated that researchers are finding contradictory and non-reflective outcomes. All of these findings indicate that there is a need for a common definition and model of user trust in health websites. While trust in health websites may be different from, say, trust in eCommerce, it is important to work towards a unification of this definition in both fields. When implementing and deploying content, such as a health infomediary, the understanding of trust over all related domains is paramount. Future work in the area of trust in health websites should build off of the 49 papers in this review. Additionally, researchers should

evaluate and consider how the constructs of trust and credibility are specific to the domain of health, instead of trying to amalgamate research from related fields. Work could also be done to apply the methods used in this paper to the research area of trust in health/medical informatics as a broader field. Last, research is emerging that examines trust as part of security, which is a growing area of research. Understanding how the field of security might be affecting the use and definition of trust will be valuable for future work.

[11] El-Attar, T., J. Gray, S.N. Nair, R. Ownby and S.J. Czaja. Older adults and internet health information seeking. in 49th Annual Meeting of the Human Factors and Ergonomics Society, HFES 2005, Sep 26-30 2005. 2005. Santa Monica, CA 904061369, United States: Human Factors an Ergonomics Society Inc. [12] Eysenbach, G., J. Powell, O. Kuss and E.-R. Sa, Empirical Studies Assessing the Quality of Health Information for Consumers on the World Wide Web: A Systematic Review. JAMA, 2002. 287(20): p. 2691-2700. [13] Fisher, J., F. Burstein, K. Lynch and K. Lazarenko, "Usability plus usefulness = trust": an exploratory study of Australian health web sites. Internet Research, 2008. 18(5): p. 477-498. [14] Flanagin, A.J. and M.J. Metzger, The role of site features, user attributes, and information verification behaviors on the perceived credibility of web-based information. New Media & Society, 2007. 9(2): p. 319-342. [15] Fox, S. and S. Jones (2009) The Social Life of Health Information, Pew Research Center’s Internet & American Life Project Web 2.0, Available from: http://bit.ly/WKgoj. [16] Fruhling, A.L. and S.M. Lee, The influence of user interface usability on rural consumers' trust of e-health services. International Journal of Electronic Healthcare, 2006. 2(4): p. 30521. [17] Glenton, C., E.S. Nilsen and B. Carlsen, Lay perceptions of evidence-based information - a qualitative evaluation of a website for back pain sufferers. Bmc Health Services Research, 2006. 6. [18] Grabner-Krauter, S. and E.A. Kaluscha, Empirical research in on-line trust: a review and critical assessment. International Journal of Human-Computer Studies, 2003. 58(6): p. 783-812. [19] Greenhalgh, T., G.W. Wood, T. Bratan, K. Stramer and S. Hinder, Patients' attitudes to the summary care record and HealthSpace: Qualitative study. British Medical Journal, 2008. 336(7656): p. 1290-+. [20] Halkias, D., N. Harkiolakis, P. Thurman and S. Caracatsanis, Internet use for health-related purposes among Greek consumers. Telemedicine Journal and E-Health, 2008. 14(3): p. 255-260. [21] Hall, M.A., E. Dugan, B. Zheng and A.K. Mishra, Trust in physicians and medical institutions: what is it, can it be measured, and does it matter? The Milbank Quarterly, 2001: p. 613-639. [22] Hesse, B.W., D.E. Nelson, G.L. Kreps, R.T. Croyle, N.K. Arora, B.K. Rimer, et al., Trust and sources of health information - The impact of the Internet and its implications for health care providers: Findings from the first Health Information National Trends Survey. Archives of Internal Medicine, 2005. 165(22): p. 2618-2624. [23] Hong, T., The influence of structural and message features on Web site credibility. Journal of the American Society for Information Science and Technology, 2006. 57(1): p. 114-127. [24] Hovland, C., I.L. Janis and J.J. Kelley, Communication and Persuasion. 1953, New Haven, CT: Yale University Press. [25] Huh, J., D.E. DeLorme and L.N. Reid, Factors affecting trust in on-line prescription drug information and impact of trust on behavior following exposure to DTC advertising. Journal of Health Communication, 2005. 10(8): p. 711-731.

6. Acknowledgements
This publication was supported by grant 1UL1RR025011 from the Clinical & Translational Science Award (CTSA) program of the National Center for Research Resources National Institutes of Health and by NSF Awards CCF-0830569 and #0851774. The University of Wisconsin-Madison Systems Engineering Initiative for Patient Safety (SEIPS) provided support on this project http://cqpi.engr.wisc.edu.

7. References
[1] Anderson, L.A. and R.F. Dedrick, Development of the Trust in Physician scale: a measure to assess interpersonal trust in patient-physician relationships. Psychological Reports, 1990. 67(3 Pt 2): p. 1091-1091. [2] Ayantunde, A.A., N.T. Welch and S.L. Parsons, A survey of patient satisfaction and use of the internet for health information. International Journal of Clinical Practice, 2007. 61(3): p. 458-462. [3] Bates, B.R., S. Romina, R. Ahmed and D. Hopson, The effect of source credibility on consumers' perceptions of the quality of health information on the Internet. Medical Informatics and the Internet in Medicine, 2006. 31(1): p. 45-52. [4] Benotsch, E.G., S. Kalichman and L.S. Weinhardt, HIVAIDS Patients' Evaluation of Health Information on the Internet: The Digital Divide and Vulnerability to Fraudulent Claims. Journal of Consulting and Clinical Psychology, 2004. 72(6): p. 1004-1011. [5] Borzekowski, D.L.G. and V.I. Rickert, Adolescent cybersurfing for health information - A new resource that crosses barriers. Archives of Pediatrics & Adolescent Medicine, 2001. 155(7): p. 813-817. [6] Chang, H.H. and C.S. Chang, An assessment of technologybased service encounters & network security on the e-health care systems of medical centers in Taiwan. Bmc Health Services Research, 2008. 8. [7] Cima, R.R., K.J. Anderson, D.W. Larson, E.J. Dozois, I. Hasson, W.J. Sandborn, et al., Internet use by patients in an inflammatory bowel disease specialty clinic. Inflammatory Bowel Diseases, 2007. 13(10): p. 1266-1270. [8] Dart, J., The internet as a source of health information in three disparate communities. Australian Health Review, 2008. 32(3): p. 559-569. [9] Dutta-Bergman, M., Trusted online sources of health information: differences in demographics, health beliefs, and health-information orientation. Journal of Medical Internet Research, 2003. 5(3). [10] Ebert, T.A.E., Interdisciplinary trust meta-analysis. Retrieved June, 2007. 25: p. 2007-2007.

[26] Huntington, P., D. Nicholas, B. Gunter, C. Russell, R. Withey and P. Polydoratou, Consumer trust in health information on the web. Aslib Proceedings, 2004. 56(6): p. 373-382. [27] Ivanitskaya, L., I. O'Boyle and A.M. Casey, Health information literacy and competencies of information age students: Results from the interactive online Research Readiness Self-Assessment (RRSA). Journal of Medical Internet Research, 2006. 8(2). [28] Jian, J.-Y., A.M. Bisantz and C.G. Crury, Foundations for an Empirically Determined Scale of Trust in Automated Systems. International Journal of Cognitive Ergonomics, 2000. 4(1): p. 5371. [29] Kaelber, D.C., A.K. Jha, D. Johnston, B. Middleton and D.W. Bates, A Research Agenda for Personal Health Records (PHRs). J Am Med Inform Assoc, 2008: p. M2547-M2547. [30] Kelton, K., K.R. Fleischmann and W.A. Wallace, Trust in digital information. Journal of the American Society for Information Science and Technology, 2008. 59(3): p. 363-374. [31] Khoo, K., P. Bolt, F.E. Babl, S. Jury and R.D. Goldman, Health information seeking by parents in the Internet age. Journal of Paediatrics and Child Health, 2008. 44(7-8): p. 419-423. [32] Komiak, S.X. and I. Benbasat, Understanding Customer Trust in Agent-Mediated Electronic Commerce, Web-Mediated Electronic Commerce, and Traditional Commerce. Information Technology and Management, 2004. 5(1): p. 181-207. [33] LaCoursiere, S.P., M.T. Knobf and R. McCorkle, Cancer Patients' Self-Reported Attitudes About the Internet. Journal of Medical Internet Research, 2005. 7(3). [34] Larsson, M., A descriptive study of the use of the Internet by women seeking pregnancy-related information. Midwifery, 2009. 25(1): p. 14-20. [35] Lee, C.J. and R.C. Hornik, Physician Trust Moderates the Internet Use and Physician Visit Relationship. Journal of Health Communication, 2009. 14(1): p. 70-76. [36] Lee, J.D. and K.A. See, Trust in Automation: Designing for Appropriate Reliance. Human Factors, 2004. 46(1): p. 50-80. [37] Lemire, M., G. Paré, C. Sicotte and C. Harvey, Determinants of Internet use as a preferred source of information on personal health. International Journal of Medical Informatics, 2008. 77(11): p. 723-734. [38] Lumsden, J. and L. MacKay. How Does Personality Affect Trust in B2C e-Commerce? in Proceedings of the 8th International Conference on Electronic Commerce (ICEC’2006). 2006. Fredericton, New Brunswick, Canada. [39] Marshall, L.A. and D. Williams, Health information: does quality count for the consumer? How consumers evaluate the quality of health information materials across a variety of media. Journal of Librarianship and Information Science, 2006. 38(3): p. 141-156. [40] Mayer, R.C., J.H. Davis and D. Schoorman, An Integrative Model of Organizational Trust. The Academy of Management Review, 1995. 20(3): p. 709-734. [41] McKnight, H.D. and V. Choudhury, Distrust and trust in B2C e-commerce: do they differ? Proceedings of the 8th international conference on Electronic commerce: The new ecommerce: innovations for conquering current barriers, obstacles

and limitations to conducting successful business on the internet, 2006. 156: p. 482-491. [42] Menon, A., A. Ddeshpande, M. PerriIii and G. Mzinkhan, Trust in online prescription drug information among Internet users: The impact on information search behavior after exposure to direct-to-consumer advertising. Health Marketing Quarterly, 2002. 20(1): p. 17-17. [43] Montague, E.N.H., B.M. Kleiner and W.W. Winchester Iii, Empirically understanding trust in medical technology. International Journal of Industrial Ergonomics, 2009. [44] Montague, E.N.H., B.M. Kleiner and W.W. Winchester Iii, Empirically understanding trust in medical technology. International Journal of Industrial Ergonomics, 2009. 39(4): p. 628-634. [45] Montague, E.N.H., B.M. Kleiner and W.W. Winchester Iii, Empirically understanding trust in medical technology. International Journal of Industrial Ergonomics, 2009. In Press, Corrected Proof. [46] Muir, B.M., Trust in automation: Part I. Theoretical issues in the study of trust and human intervention in automated systems. Ergonomics, 1994. 37(11): p. 1905-1922. [47] Newnham, G.M., W.I. Burns, R.D. Snyder, A.J. Dowling, N.F. Ranieri, E.L. Gray, et al., Attitudes of oncology health professionals to information from the Internet and other media. Medical Journal of Australia, 2005. 183(4): p. 197-200. [48] Newnham, G.M., W.I. Burns, R.D. Snyder, A.J. Dowling, N.F. Ranieri, E.L. Gray, et al., Information from the Internet: attitudes of Australian oncology patients. Internal Medicine Journal, 2006. 36(11): p. 718-723. [49] Pasmore, W.A., Designing Effective Organizations: The Sociotechnical Systems Perspective. 1988: John Wiley & Sons. [50] Powell, J. and A. Clarke, Internet information-seeking in mental health - Population survey. British Journal of Psychiatry, 2006. 189: p. 273-277. [51] Rains, S.A., Perceptions of traditional information sources and use of the world wide web to seek health information: Findings from the Health Information National Trends Survey. Journal of Health Communication, 2007. 12(7): p. 667-680. [52] Rains, S.A. and C.D. Karmikel, Health information-seeking and perceptions of website credibility: Examining Web-use orientation, message characteristics, and structural features of websites. Computers in Human Behavior, 2009. 25(2): p. 544553. [53] Rosenbaum, S.E., C. Glenton and J. Cracknell, User experiences of evidence-based online resources for health professionals: User testing of The Cochrane Library. Bmc Medical Informatics and Decision Making, 2008. 8. [54] Rosenvinge, J.H., S. Laugerud and P. Hjortdahl, Trust in health Websites: A surevey among Norwegian Internet users. Journal of Telemedicine and Telecare, 2003. 9(3): p. 161-166. [55] Rotter, J.B., Interpersonal trust, trustworthiness, and gullibility. American Psychologist, 1980. 35(1): p. 1-7. [56] Semere, W., H.L. Karamanoukian, M. Levitt, T. Edwards, M. Murero, G. D'Ancona, et al., A pediatric surgery study: Parent usage of the Internet for medical information. Journal of Pediatric Surgery, 2003. 38(4): p. 560-564.

[57] Shapiro, S.P., The Social Control of Impersonal Trust. The American Journal of Sociology, 1987. 93(3): p. 623-658. [58] Sillence, E., P. Briggs, L. Fishwick and P. Harris. Trust and mistrust of online health sites. in 2004 Conference on Human Factors in Computing Systems - Proceedings, CHI 2004, Apr 2429 2004. 2004. New York, NY 10036-5701, United States: Association for Computing Machinery. [59] Sillence, E., P. Briggs, L. Fishwick and P. Harris. Guidelines for developing trust in health websites. in Special interest tracks and posters of the 14th international conference on World Wide Web. 2005. Chiba, Japan: ACM. [60] Sillence, E., P. Briggs, P. Harris and L. Fishwick, A framework for understanding trust factors in web-based health advice. International Journal of Human-Computer Studies, 2006. 64(8): p. 697-713. [61] Sillence, E., P. Briggs, P. Harris and L. Fishwick, Going online for health advice: Changes in usage and trust practices over the last five years. Interacting with Computers, 2007. 19(3): p. 397-406. [62] Sillence, E., P. Briggs, P. Harris and L. Fishwick, Health Websites that people can trust - the case of hypertension. Interacting with Computers, 2007. 19(1): p. 32-42. [63] Sillence, E., P. Briggs, P.R. Harris and L. Fishwick, How do patients evaluate and make use of online health information? Social Science & Medicine, 2007. 64(9): p. 1853-1862. [64] Smith, M.A., B. Shneiderman, N. Milic-Frayling, E.M. Rodrigues, V. Barash, C. Dunne, et al. Analyzing (social media) networks with NodeXL. in Proceedings of the fourth international conference on Communities and technologies. 2009: ACM New York, NY, USA. [65] Song, J. and F.M. Zahedi, Trust in health infomediaries. Decis. Support Syst., 2007. 43(2): p. 390-407.

[66] Taylor, M.R.G., A. Alman and D.K. Manchester, Use of the internet by patients and their families to obtain genetics-related information. Mayo Clinic Proceedings, 2001. 76(8): p. 772-776. [67] Theng, Y.-L. and E.-S. Soh. An Asian study of healthcare Web portals: Implications for healthcare digital libraries. in 8th International Conference on Asian Digital Libraries, ICADL 2005, Dec 12-15 2005. 2005. Heidelberg, D-69121, Germany: Springer Verlag. [68] Thom, D.H. and B.B. Campbell, Patient‐physician trust: an exploratory study. Journal of Family Practice, 1997. 44(2): p. 169(8). [69] Tio, J., A. LaCaze and W.N. Cottrell, Ascertaining consumer perspectives of medication information sources using a modified repertory grid technique. Pharmacy World & Science, 2007. 29(2): p. 73-80. [70] Tung, F.-C., S.-c. Chang and C.-M. Chou, An extension of trust and TAM model with IDT in the adoption of the electronic logistics information system in HIS in the medical industry. International Journal of Medical Informatics, 2008. 77(5): p. 324335. [71] Wainstein, B.K., K. Sterling-Levis, S.A. Baker, J. Taitz and M. Brydon, Use of the Internet by parents of paediatric patients. Journal of Paediatrics and Child Health, 2006. 42(9): p. 528-532. [72] Walther, J.B., Z.M. Wang and T. Loh. The effect of top-level domains and advertisements on health Web site credibility. 2004. [73] Winkelman, W.J., K.J. Leonard and P.G. Rossos, PatientPerceived Usefulness of Online Electronic Medical Records: Employing Grounded Theory in the Development of Information and Communication Technologies for Use by Patients Living with Chronic Illness. Journal of the American Medical Informatics Association, 2005. 12(3): p. 306-314. [74] Zahedi, F. and J. Song, Dynamics of trust revision: Using health infomediaries. Journal of Management Information Systems, 2008. 24(4): p. 225-248.

Sign up to vote on this title
UsefulNot useful