You are on page 1of 27

Western Journal of Nursing Research

http://wjn.sagepub.com The Challenges of Searching for and Retrieving Qualitative Studies
Julie Barroso, Claudia J. Gollop, Margarete Sandelowski, Janet Meynell, Patricia F. Pearce and Linda J. Collins West J Nurs Res 2003; 25; 153 DOI: 10.1177/0193945902250034 The online version of this article can be found at: http://wjn.sagepub.com/cgi/content/abstract/25/2/153

Published by:
http://www.sagepublications.com

On behalf of:
Midwest Nursing Research Society

Additional services and information for Western Journal of Nursing Research can be found at: Email Alerts: http://wjn.sagepub.com/cgi/alerts Subscriptions: http://wjn.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav Citations http://wjn.sagepub.com/cgi/content/refs/25/2/153

Downloaded from http://wjn.sagepub.com at UNIV OF TEXAS AUSTIN on November 17, 2009

Western Journal of Nursing Research, 2003, 25(2), 153-178
Western Journal of Nursing Research March 2003, Vol. 25, No. 2 DOI: 10.1177/0193945902250034

ARTICLE

The Challenges of Searching for and Retrieving Qualitative Studies1
Julie Barroso Claudia J. Gollop Margarete Sandelowski Janet Meynell Patricia F. Pearce Linda J. Collins

The authors’purpose in this article is to report the results of their search and retrieval efforts to date in an ongoing study to develop the procedural, analytic, and interpretive techniques to conduct qualitative meta-synthesis projects, using studies on women with HIV infection as the method case. For researchers conducting qualitative meta-synthesis projects, the ideal goal is to retrieve all of the relevant studies in a field—not simply a sample of them. Bates’s model of berrypicking is used as the framework to describe the techniques used to conduct these searches. The authors discuss, in particular, the challenges of working with bibliographic databases, including choosing which databases to search, learning about the idiosyncrasies of working with each database, developing a list of search terms, and refining inclusion criteria regarding which studies to include in the meta-synthesis. Recommendations are given for searchers and writers of qualitative research. Keywords: qualitative meta-synthesis; qualitative research; research integration; information retrieval; information-seeking behavior; computer literature searching

There has been an unprecedented proliferation of qualitative studies on various aspects of health, illness, and life transitions across the disciplines
Julie Barroso, Ph.D., A.N.P., C.S., Assistant Professor, School of Nursing, University of North Carolina at Chapel Hill; Claudia J. Gollop, Ph.D., Associate Professor, School of Information and Library Science, University of North Carolina at Chapel Hill; Margarete Sandelowski, Ph.D., R.N., F.A.A.N., Cary C. Boshamer Distinguished Professor of Nursing, School of Nursing, University of North Carolina at Chapel Hill; Janet Meynell, M.S.N., R.N., School of Nursing, University of North Carolina at Chapel Hill; Patricia F. Pearce, RN, MSN, CS-FNP, School of Nursing, University of North Carolina at Chapel Hill; Linda J. Collins, M.S.L.S., User Services Librarian, Health Sciences Library, University of North Carolina at Chapel Hill.
DOI: 10.1177/0193945902250034 © 2003 Sage Publications 153

Downloaded from http://wjn.sagepub.com at UNIV OF TEXAS AUSTIN on November 17, 2009

Qualitative meta-synthesis is a distinctive kind of research integration study in which the findings from completed qualitative studies in a target area are combined. 1997. including. The aim of qualitative meta-synthesis is instead to create larger interpretive renderings of all of the studies examined in a target domain that remain faithful to the interpretive rendering in each particular study. This challenge is especially compelling for researchers conducting qualitative meta-synthesis projects in which the ideal goal is to retrieve all of the relevant studies in a field— not simply a sample of them. most notably.sagepub. These studies contain important findings about a diverse range of health issues. 10) and a shared desire to use a systematic. integrated. 1988. prevention. Kearney.and end-oflife technological interventions. living with and managing the effects (including the treatment effects) of an array of chronic conditions. and grounded theory. 1981) in that there is “a shared interest in synthesizing empirical studies” (Noblit & Hare. 1994. Downloaded from http://wjn. 1986. and it is a field of great significance to women’s health and nursing practice. 1996. McGraw. ethnography. 1997). p. Docherty. The most important threat to the validity of any research integration effort is to fail to conduct a sufficiently exhaustive search (Cooper. or otherwise put together (Estabrooks. and communicable approach to research integration. personal and cultural constructions of disease.154 Western Journal of Nursing Research and health-related publication venues. Crooks. 1998). Noblit & Hare. analytic. the term qualitative meta-synthesis refers to both an interpretive product and the analytic processes by which the findings of studies are aggregated. Jensen & Allen. The aim of qualitative meta-synthesis is to create larger interpretive renderings of all of the studies examined in a target domain that remain faithful to the interpretive rendering in each particular study. summarized. 1998. p. 1988. Schrieber. Sandelowski. The challenge for qualitative researchers now is no longer finding places to publish their work but rather finding all of the studies that are relevant to producing their work. & Stern. Our purpose in this article is to report the results of our own search and retrieval efforts to date in an ongoing study to develop the procedural. and interpretive techniques to conduct qualitative meta-synthesis projects. Field. & Emden. and decision making on and responses to beginning. comprehensive.2 We chose this area of research because a sufficient number of qualitative studies exist to warrant meta-synthesis. & Morse. 33). Although it can be considered an analog to meta-analysis (Glass. Like phenomenology. using studies on women with HIV infection as the method case. and risk.com at UNIV OF TEXAS AUSTIN on November 17. 2009 . qualitative meta-synthesis is not about averaging or reducing findings to a “common metric” (Wolf. & Smith.

in any kind of research. 2001). The authors of another recent article on the challenges of retrieving qualitative research noted that MEDLINE does not include the term “qualitative” as a MeSH term (Dixon-Woods. dissertations. and temporal boundaries for study.March 2003.com at UNIV OF TEXAS AUSTIN on November 17. 25. No. and changing information technology will permit. They commented that they had learned of another published meta-ethnography of diabetes only after their own study was completed and that the authors of this other report had identified only three out of the seven papers that they themselves had included in their search.” 2001). MeSH (Medical Subject Headings) terms are the controlled vocabulary of words and phrases used to index much of the biomedical literature. Evans (2002) also noted the difficulties in searching electronic databases for qualitative research. Near the end of the first year of our project. & Roberts. This sadly highlights the imperative. expanding information. Fitzpatrick. 2009 . and conference papers that will ultimately comprise the bibliographic sample for the meta-synthesis study. a healthy young woman died while participating in a clinical trial at a major medical center. Vol. on searching for and retrieving the published articles and book chapters. in press) noted the difficulties they encountered when trying to retrieve all of the literature relevant to their work. we focused. In a recent report of a meta-ethnography of qualitative research on lay experiences of diabetes and diabetes care. thereby hindering retrieval of qualitative research. to know what others have published.sagepub. 2 155 In the first year of our meta-synthesis project. He missed some reports of safety problems published in the 1950s (“Death at the Hands of Science. We describe here some of the challenges we encountered in searching for and retrieving qualitative studies and offer recommendations for both searchers and writers of qualitative studies that will make this work as fruitful and efficient for their purposes as their resources. in part. The New York Times reported that the chief researcher of this clinical trial had searched the scientific literature and found no evidence that the drug given to her might be unsafe. HOW WE BEGAN A metasynthesis project typically begins by defining the substantive. as well as the unpublished theses. the authors (Campbell et al. methodological. We began our study in June 2000 with the following inclusion criteria: all reports of qualitative studies on women in the United States with HIV infection published between 1993—the year in which the first of these studies appeared as Downloaded from http://wjn..

we expected to find approximately 35 studies. understood. both in terms of types of sources to use and search techniques to employ with these resources. We also excluded journalistic or other nonresearch accounts of HIV-positive women. 1996). 2009 . 2003. we truncated the terms to broaden the search. including theses and dissertations. discourse analysis. case study. hermeneutic. descriptive study. We created a list of method search terms that we thought would reflect our criteria well and would therefore capture the studies we were seeking. constant comparison. Browsing can be distinguished from berrypicking in that browsing is less directed and may be useful to stimulate a searcher’s thinking. We specifically excluded: (a) qualitative studies in which there are no human participants per se (as. friends. reflexive. for example. interview. content analysis.sagepub. BROWSING AND BERRYPICKING As Bates (1989) pointed out. (b) qualitative studies about nonseropositive women and their experiences as mothers.com at UNIV OF TEXAS AUSTIN on November 17. With this model. the searcher has a more complex search environment to consider. 2 years before the anticipated end of the metasynthesis project in 2005. or other qualitative analyses of media representations of women with HIV infection). conversation analysis. and case-oriented. Bates (1989) argued that berrypicking is closer to the actual behavior of searchers than traditional models of information retrieval. grounded theory. experienced. and (c) qualitative adjuncts (such as open-ended questions at the end of a structured questionnaire) to largely quantitative studies. feminist. They included. semiotic. in alphabetical order.156 Western Journal of Nursing Research indicated by our search at that time (we have since found an article published in 1991)—and March 1. naturalistic study. but browsing can be an important component of several of the berrypicking strategies we describe below. partners. participant observation. and produced (Mason. Where appropriate. relatives. exploratory study. At the time of grant writing. qualitative content. and/or caregivers of seropositive persons. in discourse. and thematic analysis. qualitative research. narrative/narrative analysis. focus group. as more resources are brought online. We defined qualitative research as encompassing a highly diverse array of orientations to and strategies for inquiry that generally concern themselves with how the social world is interpreted. Data generation and analysis techniques are flexible. ethnography. context-sensitive. qualitative method. phenomenology. considered a classic among information Downloaded from http://wjn.

600 citations from the reference lists of research reports. We have not yet done this type of searching as we wanted to have most of what we anticipate will be the final sample prior to completing this step. in part or whole. Bates (1989) identified six strategies that are used in berrypicking. and thus leaps forward (Bates. is where the searcher begins with a citation.com at UNIV OF TEXAS AUSTIN on November 17. We focus in particular on subject searches in bibliographies and abstracting and indexing services. the searcher wanders through the information forest. The query itself is continually shifting. It usually takes some time for an article to disseminate widely and therefore to be cited by others in more recent works. We searched the reference lists for anything that was remotely connected to women and HIV infection. and the one most commonly used by searchers. Footnote Chasing This technique includes following up on footnotes found in books and articles of interest and therefore moving backward through reference lists. we describe these now along with how we actually executed these strategies during our search and retrieval work to obtain qualitative studies of women with HIV infection. linear process. we reviewed more than 3. changing directions as needed to follow up on various leads and shifts in thinking. In our project. even if the actual document being searched was not a research report that might be included in the meta-synthesis. 25. books. This is called an evolving search and is closer to the actual behavior of searchers. at each stage of the search. We also surmised that this may not be a very fruitful technique for us. The query is satisfied not by a single final retrieved set but by a series of selections of individual references and bits of information at each stage of the ever-changing search. Just as a berrypicker would meander through the bushes. Downloaded from http://wjn. given how recently most of the articles we are using were published. and anthologies on women with HIV infection. 2009 . This is not a straight. 1989).March 2003. finds out who cites it by looking it up in a citation database. Vol. 1989). Citation Searching Citation searching. as this was the most complex strategy. the searcher is not just modifying search terms to get a better match for a single query. looking for clumps of berries (Bates. 1989). No.sagepub. and other articles. 2 157 scientists. it is also called backward chaining (Bates. or forward chaining.

Of the authors whose published work(s) we had already retrieved. Qualitative Health Research and the Journal of the Association of Nurses in AIDS Care. that were not also found using bibliographic databases. searching the electronic card catalog of our library system looking for library materials that could be found by using “women and HIV” as our subject search terms. we searched by each author name for dissertations.com at UNIV OF TEXAS AUSTIN on November 17. once the researchers have identified a central journal in an area. We also searched for articles in every database using each author’s name to see if we had not yet retrieved a published article. We have had good results using this technique. Author Searching Based on relevant publications found previously. we scanned the shelf locations of these books to find other books that might be of interest to us. using this journal-run technique. were the closest to being central. 1989). and as a widely used and effective technique. These two journals. Downloaded from http://wjn. We felt that the first is an excellent source for qualitative work. these journals are systematically hand-searched straight through the relevant volume years (Bates. Once we had identified several books that dealt with HIV-positive women.sagepub. We supplemented area scanning with electronic searches as well. We located several dissertations on women with HIV infection using this method. particularly in the iterative searching for dissertations and/or the articles generated from those dissertations. as many of the researchers in the area of women and HIV infection are recent graduates of doctoral programs. 2009 . Such a technique guarantees complete recall within that journal. 1989). this strategy entails searching by author name to see if an author has done any other work on the same topic (Bates. We have done this with two journals. We did not find any articles in either of these journals. although not precisely central to the area. Area Scanning Bates (1989) described this technique as browsing the materials that are physically colocated with materials located earlier in a search.158 Western Journal of Nursing Research Journal Runs With this technique. and the second is a good source for research conducted with HIV-positive people and is the flagship journal of the Association of Nurses in AIDS Care.

We chose bibliographic databases that were accessible to us.g. SilverPlatter (Silver Platter Information). Searches may be categorized into two forms based on precision and recall: high-recall searches.). Sutcliffe. No. 25. given a set of retrieved documents. and the system(s) involved (e. Inc. Marchionini. Our access to these databases was through the library computer network at the University of North Carolina at Chapel Hill. for a total yield of 3. 1994. often consisting of a small number of documents. (text continues on p. Walker. Most citations were found in more than one database. The yield of a search is highly related to the task complexity. Haynes. has been the one we have focused on and will be discussed in greater detail. McKibbon. 2000. and high-precision searches. Marchionini. 1995). along with the features of each of these databases. and these numbers represent electronic searches only. & Watkinson. in which the set of documents to be retrieved. 2000. 2009 .. Table 2 illustrates the yield of the bibliographic database searches we have completed to date. that covered a wide range of disciplines. Boyce. Wilczynski.sagepub. The precision. is expected to be composed of predominantly relevant documents (Losee. approximately twice the number of studies we expected to find. in which most or all of the documents on a topic should be retrieved. The servers we used were primarily OVID (Ovid Technologies). specifying the number of citations that met the search terms (5. 2000). & Kraft. which involves searching bibliographic databases for relevant works in an area. We identified 16 bibliographic databases for our search. is the percentage of documents that have been retrieved that are relevant. 165) Downloaded from http://wjn. and that we knew or surmised would yield reports of qualitative studies on women with HIV infection.829) and the number of citations that actually met the criteria for inclusion into our bibliographic sample (213). and OCLC FirstSearch (OnLine Computer Library Center.com at UNIV OF TEXAS AUSTIN on November 17. the user. & Sinclair. Ennis. The most commonly used performance measures in information retrieval are based on precision and recall. Meadow. Vol. We have thus far been unsuccessful in obtaining the proceedings from three national conferences on women with HIV infection. 2 159 Subject Searches in Bibliographies and Abstracting and Indexing Services This technique. We have thus far located 71 published and unpublished studies (theses and dissertations) that meet our inclusion criteria. The recall for a set of retrieved documents is the percentage of relevant documents in the database that have been retrieved.66%. In strictly quantitative terms.March 2003. 1995. these are listed in Table 1.

special publications.800 peer-reviewed journals Yes Journals plus 1980government and present technical reports.500 journals. 2009 Database Name Academic Search Elite Content/ Kind of Information How to Access Producer or Content Provider EBSCO Publishing Thesaurus With Standardized Keywords? Yes Standardized Natural Language Language Searching for Searching Qualitative and Boolean Research? Operators? Yes Yes Special Features Dates of Coverage Scholarly. EBSCOHost multidisciplinary database covering a variety of academic areas Specialized index to AIDS-related research.National Library rated into of Medicine OVID (part of NIH) MEDLINE or PubMed Yes No Anthropological Index Online Gateway to Web articles for periodicals in the British Museum’s Mankind Library Department of Ethnography at the British Museum No No Full text for more than 19841. present covers major newspapers and more than 1. including clinical and health policy issues AIDS Information Online (AIDSLINE) Now incorpo.com at UNIV OF TEXAS AUSTIN on November 17.sagepub. AV materials and theses No—incomplete Specialized collection Late Boolean 1950spresent . meetings abstracts.160 TABLE 1: Electronic Databases Selected for Searches Downloaded from http://wjn.

in part.S.com at UNIV OF TEXAS AUSTIN on November 17.sagepub. conference proceedings. physical.Anthropological Literature Black Studies Cumulative Index to Nursing and Allied Health Literature (CINAHL) An index to periodicals and edited works— emphasis is on archaeology. Canadian dissertations. includes about 1. African. and dissertations Selectively covers master’s theses. nursing standards. anthropology (biological. AV materials. Institutions since 1861 OCLC ProQuest FirstSearch Information and Learning Yes No Yes Journals plus books. cultural. educational software. and social). on the holdings of the Schomburg Center for Research in Black Culture 1989present OVID CINAHL Information Systems Yes Yes— Yes “qualitative studies” Dissertation Abstracts Index (DAI) Index to dissertations accepted at accredited U. and African Diasporan studies Bibliographic references and abstracts on nursing and allied health topics CD-ROM Harvard University (Tozzer Library) Yes No Yes International coverage. 2009 CD-ROM Gale Group No No Yes Based. and British and other European dissertations 1982present 1861present (continued) 161 .000 journals and monographs 1985present Downloaded from http://wjn. and linguistics International index to African American.

com at UNIV OF TEXAS AUSTIN on November 17.162 TABLE 1 Downloaded from http://wjn. 2009 (continued) Producer or Content Provider Educational Resource Information Center Database Name Content/ Kind of Information How to Access Thesaurus With Standardized Keywords? Yes Standardized Natural Language Language Searching for Searching Qualitative and Boolean Research? Operators? Yes Yes Special Features Includes curricular materials such as courses of study. dissertations. and book chapters . veterinary medicine. dentistry. conference papers. and related disciplines National Library of Medicine (part of NIH) Yes No Yes International cover1966age. nursing. books. includes present approximately 4. the health care system. and the preclinical sciences PsycInfo Index to the Silverplatter research and clinical literature in psychology.600 current biomedical journals published in the United States and 70 foreign countries American Psychological Association Yes No—unless Yes author lists as “key phrase” (not a controlled descriptor) International cover1887age. and theses Dates of Coverage 1966present Educational Database of Silverplatter Resource journal and Information report literature Center (ERIC) in education and related disciplines MEDLINE Database of OVID references and abstracts covering the fields of medicine. includes present scholarly journals.sagepub. psychiatry. books.

political science. social policy.600 current biomedical journals published in the United States and 70 foreign countries. No Yes International cover1972age. related disciplines No Yes Yes Yes International coverage.Silverplatter National Associa. the health care system. and “other materials” 1977present (continued) 163 . Wilson No ences and FirstSearch Company abstracts covering anthropology. veterinary medicine. includes jourpresent nals. dissertations. Inc. and the social sciences in general PubMed Database of Web at references and Pubmed. and sociology Index to the litera. includes journals.com at UNIV OF TEXAS AUSTIN on November 17. books. and the preclinical sciences Social Science Abstracts (SocSci Abstracts) Public Affairs Yes Information Service. nursing. psychology. 2009 National Library of Medicine (part of NIH) Yes No Yes Social Work Abstracts Database of refer. Inc. and committee reports International cover1966age. abstracts covergov ing the fields of medicine. Engpresent lish-language periodicals in the social sciences Downloaded from http://wjn.Yes ture in social tion of Social work and Workers. PubMed is the MEDLINE file plus some additional citations to life sciences journals More than 350 key 1983international. geography. includes present approximately 4. statistical compilations. economics. dentistry.OCLC H. law.sagepub. government documents. W.Public Affairs Bibliographic Silverplatter Information index to the Service (PAIS) literature on public policy.

Yes theoretical and tific Abstracts applied studies covering sociology and related disciplines Interdisciplinary CD-ROM Gale Group No database index(CD edition ing approxiof the mately 100 EngWomen’s lish language Studies journals Index) No Yes International cover1963age. 2009 (continued) Producer or Content Provider Thesaurus With Standardized Keywords? Standardized Natural Language Language Searching for Searching Qualitative and Boolean Research? Operators? Yes—“qualitative methods” Yes Database Name Sociological Abstracts (Sociofile) Content/ Kind of Information How to Access Special Features Dates of Coverage Women’s Studies Database of Silverplatter Cambridge Scien. meeting abstracts. includes scholpresent arly articles. book reviews.sagepub. . and popular materials Note: NIH = National Institutes of Health. book reviews. includes more present than 2. and dissertations International cover1989age.com at UNIV OF TEXAS AUSTIN on November 17.164 TABLE 1 Downloaded from http://wjn.500 journals.

whereas others are updated only quarterly or even yearly. extinct.97 0. or incorporated into another database.29 0. were incorporated into PubMed as of September of 2001. Journal articles that could be found in AIDSLINE.96 1. variations in searchers’ activities and fatigue or Downloaded from http://wjn. Bibliographic databases are not stable entities yielding fixed search results. in addition to those formerly found in AIDSLINE.sagepub. The searcher is also not a stable. Working with and around bibliographic databases.00 2. 25. resulted in inexact recall because PubMed now indexes other health-related journals. thereby contributing to a citation list that may not be current. For example. 2009 .March 2003.05 13.91 9. No.76 6.com at UNIV OF TEXAS AUSTIN on November 17.76 4.54 9.66 our searches were not high in precision because we intentionally traded precision to heighten recall to ensure an exhaustive search. some are updated daily. once an independent database.101 452 5 1. 2 165 TABLE 2: Search Yields by Bibliographic Database Bibliographic Database Academic Search Elite AIDS Information Online Anthropological Index Online Anthropological Literature Black Studies Cumulative Index to Nursing and Allied Health Literature Dissertation Abstracts Index Educational Resource Information Center MEDLINE PsycINFO Public Affairs Information Service PubMed Social Science Abstracts Sociofile Social Work Abstracts Women’s Studies Total Citations Identified 63 338 98 338 558 504 181 54 1. Databases are updated at different intervals. PsycINFO.00 2.00 7.829 Citations for Inclusion 3 27 1 1 0 38 9 0 28 42 0 31 6 14 4 9 213 % Yield 4.02 0. and that are searched simultaneously.514 46 285 41 201 5. They may exist one day as independent entities and the next day as inaccessible. qualitative studies on women with HIV infection. Vol. unchanging entity. This change meant that searches on our topic. access to CD-ROM-based PsycLit was removed soon after we began our searches. in our library system. leaving us access to an online database.30 0.54 4.48 3.04 4.

used to further customize a search. abstract field). Two examples of the complexity of searching options that we had to master were that PsycINFO has a thesaurus or controlled vocabulary. Downloaded from http://wjn. thus maximizing the use of search terms and MeSH headings. The use of special operators (e. Each database system offered numerous options for searching. We had to become more familiar with the features of each database system.g. which is further divided into detailed subheadings used to search for more specific information. and not are placed between search terms to specify their desired relationship to each other. but some of the databases allow more flexible use than others. to help define the items that will be retrieved by a search. appropriate use of Boolean and other operators. 2009 . 2001). Expert-level searching generally entails searches that are conducted by professional search intermediaries who employ appropriate syntax.166 Western Journal of Nursing Research frustration will influence the quality and amount of time spent conducting a search and search results. or. in OVID-based systems. MEDLINE uses MeSH terms. We chose to explode our searches. title field. and we had to spend the time investigating which of them would have the best yield in terms of recall. and inclusion of desired delimiters or expanders for the search. Boolean operators allow the searcher to use set theory. One of our first challenges was to improve our skills in interacting with these database systems and to work with them to extract the information we were seeking. For instance..sagepub. Almost everyone who has searched a database is familiar with Boolean operators. or that the terms appear in the same fields (e. which allows the searcher to broaden the search and to further explore the subject matter and related subjects. a searcher can explode a search term. in a specific order. or. can request that terms appear adjacent to each other. which is commonly recognized in the form of a Venn diagram of overlapping circles that indicate the and. Each database access provider has an information page with its parameters for updates and inclusion dates. near each other. in which the words and. proximity operators) is generally available on all of the databases. these headings are the keys that unlock the medical literature (Coletti & Bleich. including thesauri and the use of special options.g. and not functions. separated by a given number of intervening words. Proximity operators.com at UNIV OF TEXAS AUSTIN on November 17.. A searcher can use Boolean or other operators in these and other databases as well. Decisions thus had to be made regarding which terms to use and how to best use them. or a searcher can focus a search term (to narrow the search) to look for citations in which the search term is a major point of the article. as well as less specific subject subheadings. such as Boolean and proximity operators.

totally irrelevant records may appear. searches for citations satisfying the criteria. Merging of several individual search results to form a comprehensive search for the phenomenon of interest is relatively friendly in OVID. Although all the databases we used had characteristics that were supportive of our searches. 1996). where more options are offered (e. If the search term is mapped to the best judgment of the programming within the database. and returns the information to the screen. this helped to reduce the number of irrelevant Downloaded from http://wjn. No. and the related mappings in that process can be viewed. Minor idiosyncrasies. 25. In mapping. Mapping is a relatively recent addition to the usual line of searching features. There are options in OVID that enable the searcher to see the hierarchical mappings and relationships of individual search terms and to select or deselect the terms. we had some control in the selection/ exclusion of a particular mapping.. Although there were commonalities among many of the databases. language).g. we learned that the vocabulary that was successful in searching for studies in one database was not necessarily successful in searching in another database.sagepub. 2009 . Once visualized. The system translates the information into the underlying computer query language in operation. the system offers additional terminology from the controlled vocabulary of a database in response to a search term entered by the user (Jasco. Where a database resides can influence searching and subsequent results. Vol. such as whether a comma should be used between an author’s last name and first name (or initial) vary substantially among databases and can be irritating to the searcher. Few database systems return a message telling searchers that they have made a mistake in entering search terms that a system’s language cannot map appropriately. The same database provided by two different vendors could conceivably return different results of the same search because searching features vary among database vendors. These are combined within the system to form what is known as the search string. in addition to providing an invalid search result. Within each search. 2 167 There were also variations within and among the databases in the mapping of the search terms. the search terms may be combined with the options selected from the main or basic database search screen.March 2003. The searcher may thus be faced with no citations or many irrelevant ones. publication year. Any search term can be expanded or focused in a variety of ways. maps the information to the appropriate areas. A no records found message may not reflect the absence of records so much as a system’s inability to map the search terms used. or terms may be entered into the advanced search window.com at UNIV OF TEXAS AUSTIN on November 17. we found OVID to be the most useful access provider/ vendor for several reasons.

We found this aspect particularly helpful when developing an appropriate search term list. our search process was necessarily more laborious. which is a paper-based directory available in most libraries that can also be accessed electronically by subscription.com at UNIV OF TEXAS AUSTIN on November 17. 1998) was cited in 12 different databases. followed by PubMed and AIDSLine. OVID provides a seemingly endless number of related subject headings and keywords that represent the indexing of each citation and provide the searcher with a rich source of terms to use. An additional source of information regarding journal indexing is Ulrich’s Periodicals Directory. Knowing where published works likely to be relevant to a project are indexed is critical to finding those works. Most of the overlap or duplicate search results may be attributed to inclusion practices of the various indexing and abstracting database producers.med. We also utilized direct links to some full text articles that are provided in jake. which each had the same 38 citations. Mueller.edu/) database at Yale University (hosted by the Cushing/Whitney Medical Library at the Yale University School of Medicine) improved our ability to identify the electronic index or indices in which a particular journal could be found. OVID also permits the complex use of nested Boolean operators and allows the user to save and e-mail the results in a form that is friendly to importing directly into a reference manager database such as ProCite or EndNotes. We used two means to track down citations and to develop our list of databases to search for the project. There was considerable overlap among these databases.yale. For example. In the case of search and retrieval. Of the 71 published articles and unpublished theses and dissertations that met our search criteria and that we have retrieved thus far. Downloaded from http://wjn. The publicly accessible Jointly Administered Knowledge Environment (“jake” at http://jake. OVID provides a clear error message when a searcher has made an error in handling search terms or applications.sagepub. Because one of the major objectives in our method project was to explore the various processes involved in conducting qualitative metasyntheses. The nature of a research project will determine the nature of the search process. Medina. Of the 71 studies. we deliberately overworked to test how much work was actually necessary to produce a valid search result. CINAHL and PsycINFO yielded the largest number of citations (43 each). 32 could also be found through the more laborious footnote chasing discussed above. Databases to find databases.168 Western Journal of Nursing Research citations that we had to sift through to find relevant works. 2009 . & Wolf. Our primary obligation in this project is to do the work required that would put us in a credible position to advise other researchers of work they need not do. one article (Dunbar.

But Ulrich’s Periodicals Directory (2000) contains many more journal titles and is considered a more comprehensive compilation of journal indexing than jake (Chudnov. HIV. phenomenology. grounded theory. 1995. Vol. abstract. daughters. 2000.March 2003. and wives. as well as battered females. This list. By utilizing words anywhere in our searches.sagepub. Ulrich’s also has information about the peer-reviewed status of indexed journals. and female criminals. including women. March 22. is not always readily apparent. we realized that these search terms had neither the specificity nor sensitivity we desired. widows. After a few trial searches.. but this gave us more confidence that we would capture more of the relevant literature.g. and the database will attempt application. Downloaded from http://wjn. the search term women maps to human female. No. sisters. Collins. 2009 . Meadow et al. females. Human female includes mothers. We initially thought the search term list would entail a relatively small number of terms. AIDS. mothers. We used the words anywhere option to ensure a comprehensive search. that is. For example. & Parker. Most databases allow the searcher to select the location to which the search terms. 2000).com at UNIV OF TEXAS AUSTIN on November 17. The development of search terms appropriate to the area of interest is essential for the effective use of bibliographic databases. keywords. 25. natural language processing allows for the searcher to type in virtually any term or phrase. there were a large number of irrelevant citations produced. 2001. in some databases (e. Crooker. We had to become more knowledgeable about the underlying mapping patterns for each database to manipulate search terms appropriately. In contrast. or words anywhere (meaning every textual piece related to a citation that is in the database). Developing a list of search terms for the meta-synthesis. which allows common or noncontrolled terms to be searched. ethnography. Marchionini. qualitative research. 2001). we knew we would capture many more citations than would be relevant. which is a common technique in information retrieval (Coletti & Bleich. 2 169 Both have cross-referencing of journals. others work by natural language processing.. author. working women. and interview. however. or phrases in the citation should be applied: for example. Still other databases are searchable using both methods. although relevant citations that we already knew of were not captured. A controlled vocabulary database means that the effective search terms in the database must be on the list within the database. L. the title. personal communication. however. Some databases can only be searched by a controlled vocabulary (or thesaurus or list of available terms). words. PsycINFO). naturalistic research.

We accordingly generated a more refined list of qualitative research search terms. If we were unsure. Qualitative research. we checked the abstract. An especially challenging problem was the difficulty locating exclusively qualitative research reports. is graphically shown in Figure 1. Uncertain citations then led to a negotiation of consensus on its status and further refinement and delineation of inclusion criteria. there were still hundreds (about 20%) of citations retrieved that could not be clearly excluded on the basis of title and abstract alone. Table 1 lists which of the 16 databases we used indexes according to research methodology. depending on the focus of the search and retrieval activity. each of these subheadings then maps to other terms. or in the full text of the article. The challenge was to use search terms and operators that spoke the language of each of the bibliographic databases while still representing qualitative research. The most important thing we realized. we also searched using the many terms that indicated that the methodology used was qualitative in nature. nursing research. We therefore developed a process to track the decisions we made about these citations. As a search term. qualitative research yielded few relevant citations and many irrelevant ones. and we truncated the terms as well to cast a wider net. Qualitative research in MEDLINE maps to a large number of subject headings: research. which deals with both substantive and methodological issues. In OVID-based CINAHL. and myocardial infarction. these mappings can be selected or deselected. qualitative research maps to qualitative studies. At that point. In OVID-based MEDLINE and CINAHL. such as qualitative laboratory assays. the citation was included.170 Western Journal of Nursing Research The database may be searching for key words as provided by the author. abstract. is that most bibliographic databases do not index articles according to research methodology. health services research. the indexer. nursing methodology research. A citation could then be excluded at this point as not meeting the criteria. data interpretation. Determining relevance.com at UNIV OF TEXAS AUSTIN on November 17. nursing. excluded. research design. or it could merit further investigation. or contained in the title. Downloaded from http://wjn. If further investigation was warranted. using the berrypicking process. 2009 . Even when we refined our search terms and techniques. we were sometimes able to exclude it based on the title alone. Once a citation was obtained. As stated earlier. This process. We therefore manipulated our search terms according to the parameters of the systems we searched. or its status remained uncertain. Following hierarchical mapping.sagepub. the full article was obtained. particularly because we are doing qualitative meta-synthesis work.

retrieval.March 2003. w/ or w/o citation document Specific Step.com at UNIV OF TEXAS AUSTIN on November 17. 25. p. Vol. Search. NOTE: Reference for graphic symbols drawn from Harris (1999. 156). 2009 . w/ citation document (electronic/paper) Study Electronic Management Database Descendency Ancestry Figure 1. Downloaded from http://wjn.sagepub. and validation process. No. 2 171 Search Term List Search Citation Database Check Title Include Unsure Exclude Check Abstract Include Unsure Exclude Check Full Article Include Unsure Exclude Consult for Consensus Include Exclude Inclusion for Bibliographic Sample Exclusion from Bibliographic Sample Check Bibliography Check Author Citations Initial/Ending Activity Decision Point Specific Step.

such as ProCite and EndNote. such as PsycINFO or Sociofile. As shown in Figure 1. and the author was searched in all of the different databases to see if he/she had published something else that met the criteria. and PAIS. In most cases. that were ultimately accepted. We located most of the theses and dissertations we were interested in through Dissertation Abstracts International (DAI). This often required obtaining copies of articles through interlibrary loan for a peritem fee. its reference list was checked for other studies that might meet our criteria.172 Western Journal of Nursing Research Once any article was finally included as part of the bibliographic sample for the meta-synthesis. The foundation for ensuring valid procedures and results is the establishment of a clear audit trail documenting all procedural moves and decision-making points. Anthropology Index.com at UNIV OF TEXAS AUSTIN on November 17. 2009 . Anthropology Literature. Reference manager software.sagepub. If the researchers cannot afford to purchase these works. The search for unpublished works can be a costly venture that some searchers may not be able to afford because most of these works would have to be purchased to determine their relevance to a project. and that required more investigation. MANAGING INFORMATION Projects such as this generate volumes of information that require efficient management and cataloging. Although we ultimately excluded many of these articles. making management of the search results highly efficient and effective. All the database systems we used supported the transfer of our search results to a printer and/or to saving them on a disk or hard drive. Our initial search of DAI yielded 8 theses and 181 dissertations. With the exception of Black Studies. we had to find a way to track citations that were removed immediately. proved invaluable to us in managing the Downloaded from http://wjn. this may contribute to a less complete bibliographic sample for meta-synthesis. Women’s Studies. For example. a full text copy was appraised for every study that was included in the meta-synthesis to ensure it met all study criteria. Searching for and retrieving unpublished theses and dissertations presented a special challenge to determining relevance. these works could not be obtained on interlibrary loan. both strategies described above in the berrypicking techniques. we considered this neither a waste of time nor money as we gained articles that will be useful to us in contextualizing the results of our meta-synthesis. OVID-provided searches are also readily transferable into ProCite or other reference manager software packages. although some of them appeared in other databases as well. all the database systems we used supported the transfer of our results through email.

1996). 25. Hjorland. 2000). CONCLUSIONS AND RECOMMENDATIONS We have tried here to fix in time and space a process that is in reality a moving target. We created several Access databases for managing those citations that required more extensive tracking and handling. we offer recommendations for searchers and writers of qualitative reports that we hope will help them manage it. can affect the validity of a search as well.March 2003. MEDLINE ) have human indexers. 2001. Meadow et al. This type of methodological filter is a predetermined search strategy that uses terms related to research design to identify all those studies using the research method of interest to the searcher. No. most database systems (e. 2 173 results of our searches. ProCite. those using qualitative methods) (Evans. and Access through several mechanisms. OTHER THREATS TO VALID SEARCHES Despite the steps we have taken to ensure a valid search. based on Evans’s (2002) assertion that the effectiveness of electronic database searchers would be improved through the development of a qualitative research search filter. Other types of errors. and as with any work performed by humans.. 71%) misspelled words occurred within the abstract only and that 98 of the 200 articles with misspelled text words might be missed if a searcher conducted a MEDLINE search using the correctly spelled word alone without the heading (Ray & Vermeulen.com at UNIV OF TEXAS AUSTIN on November 17.sagepub. We have a suggestion for indexers. however.. they can make mistakes entering information into an electronic citation database (Coletti & Bleich. 2009 . The purpose of such a filter would be to retrieve only those studies most relevant to the review (in this case. from authors and editors. The authors found that most (141/200. 2002). Vol. The publicly available PubMed system is directly searchable. and our search results were directly downloadable through our university’s ProCite license. For example. Information can be readily transferred between EndNote. 2001. Downloaded from http://wjn. such as Dissertation Abstracts. a total of 200 citations were retrieved from 10 selected search terms. Acknowledging the situational dynamic of searching for and retrieving qualitative studies. In a study that examined the accuracy of entries in MEDLINE by searching for misspelled text words.g. there are other threats to validity over which a searcher has no control.

and citation databases. and arts. in addition to those in other works about women with HIV infection. This involves configuring a number of factors.com at UNIV OF TEXAS AUSTIN on November 17. For example.174 Western Journal of Nursing Research Recommendations for Searchers Because there is a different calculus involved with every search. of these 32 studies. in hindsight. in part. The irony is that with so much information available at our fingertips. in our case. This in turn requires searchers to learn the intricacies and idiosyncrasies of working with and around these databases. including the amount of time they can expend on locating relevant works. the amount of money they can spend to retrieve full-text copies of works. and the richness of their search environment. This yielded 32 studies that were ultimately included in our sample. retrieving relevant materials has become harder. Because our research project demanded that we investigate and then recommend the best procedures to conduct qualitative meta-synthesis studies. We will not know until we are further into the meta-synthesis process itself if we will find any other studies exclusively through a hand search of a book’s chapters or the reference lists in a book or in articles. searchers must also determine which databases in these varied disciplines they must have access to so as to achieve a valid search. Because qualitative research spans disciplines in the sciences. servers. searchers must figure out the specific calculus for their particular project or topic. how to best approach the search process and whether to emphasize precision or recall. We have seen. we reviewed more than 3. but also because of the lack of standard- Downloaded from http://wjn.sagepub. Searchers must determine the resources they will have available to them for their search. because of advances in technology. We have located 1 study that appeared as a chapter in a book (Kass & Faden.600 citations appearing in the reference lists of reports we accepted into our bibliographic sample. Searchers may need to spend more time than they had anticipated enhancing their search and retrieval skills. which can be done with tutorials available on most bibliographic databases. This portion of the calculus will help searchers define their overall search strategy: that is. we deliberately chose more laborious and time-consuming search approaches to determine whether less laborious or time-consuming methods would produce the same results. the need for an information specialist on a research project such as ours. Would the studies that a meta-synthesist does not find change the results? We might find that. Yet. Of greater importance are the consequences of not finding studies by doing electronic searches alone. humanities. 2009 . all could be found electronically as well. 1996) and was not available as a citation on any of the databases we searched. including the availability of expert librarians. the time we spent in doing this ancestry work was unnecessary.

including keywords and the construction of the title and abstract. They will likely have to rethink and refine their inclusion and exclusion criteria as the search begins to yield studies. noted that although methodological search filters have worked well with locating randomized controlled trials. Searchers could then search under that term rather than having to search with terms covering every kind Downloaded from http://wjn. No. Searching in and of itself is an area of specialized knowledge. Recommendations for Writers Although we have emphasized searching and searchers here. Barry. and even these features can change with some frequency. Every bibliographic database has its own unique features. at what point they can stop searching and still meet the criterion for having conducted an exhaustive search that all research integration projects require. Searchers will also need to have the flexibility to match the ever-changing situational dynamic of the search process itself. searchers must decide at what point they will be satisfied with their results and. He noted that the lack of a clear description of the method used in the title or abstract of a study makes it difficult to find qualitative studies when doing electronic searches. 2001) if they attended more thoughtfully to those aspects of research reports that are most visible in electronic systems. Coletti & Bleich.com at UNIV OF TEXAS AUSTIN on November 17. Although constrained by publication venues and database systems. Because there is a lag time between the arbitrary end of a search and the completion of a meta-synthesis of the findings from the reports retrieved from that search. given the regular use of established thesauri and subheadings. 1998. in discussing the importance of systematic reviews for evidence-based practice. 2009 . Although more challenging to achieve.March 2003. we would also recommend that indexers use the same term to index any study conducted using any qualitative methodology..sagepub. researchers use the term qualitative research as a keyword in every report of a study using any qualitative methodologies. the unique characteristics of qualitative research limit the easy translation of these search techniques. we found that writers play a key role in determining whether their work will be found. Evans (2002).g. Because there is no fixed endpoint to searching. and metasynthesists would be advised to have an information specialist as part of the team. researchers must decide whether they will update their search before submitting their work for publication. we recommend that. whenever possible. For example. more important. writers could contribute to the retrievability of their work and to research dissemination and utilization (e. Vol. 25. 2 175 ization across database systems.

searchers may immediately exclude it from consideration. Downloaded from http://wjn. 1293-1303. L. 13. (1989). Electronic environments raise searchers’ expectations for information access. be halted. 2. there is no argument that any meta-synthesis of qualitative findings rests on a sufficiently exhaustive search. Writers should also be aware that searchers have a heightened expectancy with bibliographic databases. titled “Analytic Techniques for Qualitative Metasynthesis. and comprehend information if they are to achieve their goals (Marchionini. Journal of the American Society for Information Science. the University of North Carolina at Chapel Hill. writers need to do everything they can to ensure that they are providing appropriate terms to aid searchers. But these expectations interact with searchers’ physiology and psychology.com at UNIV OF TEXAS AUSTIN on November 17. Searchers make critical decisions about whether to investigate citations further based on what is in front of them on a computer screen. process. 1995). Blackwell and Lynne D. NOTES 1. Searchers become impatient while waiting even a few seconds because of the fundamental change in temporal expectations fostered by computers. research dissemination. In summary. J. 2009 . Bates. Document representations and clues to document relevance. C. Searchers must still perceive. This study. Online Review. although there is still much debate on what qualitative metasynthesis projects entail and how they should be conducted. If a citation appears immediately to be irrelevant. (1998). Morris. 49. Qualitative research presents both searchers and writers of qualitative reports many challenges that must be met to make the best and the most of the qualitative findings that offer so much to the health and wellbeing of the people nurses serve. M. R01 NR04907 from the National Institute of Nursing Research.176 Western Journal of Nursing Research of qualitative method or technique.” is supported by grant no.sagepub. therefore. Writers need to know this to ensure that what is relevant about their work will be as immediately accessible to the searcher as possible. and research utilization may also. Because one of the goals of research is to disseminate the findings to an audience. The authors acknowledge the assistance of Jean C. and the cycle of research. 407-424. REFERENCES Barry. The design of browsing and berrypicking techniques for online search interface. Health Sciences Library.

. R. McKibbon. 90. (1996). G. Toward a theory of aboutness. Morgan. 774-778. CA: Sage. Journal of the American Society for Information Science. private lives (pp.. Database searches for qualitative research. Marchionini. Psychological and spiritual growth in women living with HIV. HIV.. 7. C. Kearney. H. 313. British Medical Journal. Information seeking in electronic environments. London: Sage. (1994). 25.. D.. Online. CA: Sage. C. A.. Death at the hands of science. 179-186. New York: Cambridge University Press. C. R. Synthesizing research. No. Kass (Eds. Text information retrieval systems. 2009 . Jasco. 2 177 Campbell. D. M. Beverly Hills. . Harris. C. B. In R. Pound. 12-17. R.March 2003. Britten. (2001). Pill. Field. Medina. M. T.. Medical subject headings used to search biomedical literature. C. & Parker. (1995). Cooper. Thousand Oaks. Ray. Kass. J. & Vermeulen. & Faden. New York: Oxford University Press. (2000). Meta-synthesis of qualitative findings. W. L. N.. Wilczynski. 52. (1996).. M. & Morse. H. C. McGraw. Meadow. A. (2000). H. H. (1996). D. Pope. Jensen. (1996). 125-133. K. & Roberts. Haynes. (2002). Noblit.... July 31). domain. Meta-analysis in social research. Coletti.. W.. Meta-ethnography synthesizing qualitative studies. 4. Aggregating qualitative findings: An approach to theory development. A. Research in Nursing & Health. H. Glass.. 8.. Downloaded from http://wjn. R. Social Science and Medicine.. Qualitative Health Research. & Hare. Mueller. Qualitative Health Research.. Serials Review. L. Journal of the American Society for Information Science and Technology. 553-560. (2000). & Wolf. J. 503-511. Including qualitative research in systematic reviews: Opportunities and problems. Journal of Evaluation in Clinical Practice. A. B. Journal of the American Medical Informatics Association. & Allen. Ready-to-wear: Discovering grounded formal theory. J.. R. (1999). M. H. N. Journal of the Medical Librarians Association.nytimes. Hjorland. N. B. 26(4). G. K. 426-446). Mason. subject topicality. Dunbar. Evaluating meta-ethnography: A synthesis of qualitative research on lay experiences of diabetes and diabetes care. New York: Oxford University Press.. & Smith. (1994). M. San Diego. R. In women’s words: The values and lived experiences of HIVinfected women. M. T. J. C. R. 24-30. AIDS and childbearing: Public policy. (2001). CA: Academic Press. Boyce. T. Ovid web gateway: Nobody does it better. 290-293.. 834-840. and relevance.. G. P. . 43(2). 21.com at UNIV OF TEXAS AUSTIN on November 17. Social Work: Journal of the National Association of Social Workers. 1(6). 317-323. N. Chudnov. (1998). CA: Sage. (1988).. J. Jake: Overview and status report. L. Journal of the American Medical Informatics Association.). (1996). 447-458.. words: Mizspellin and MEDLINE.. M. (2001). P. Vol.. G. D. Walker. Developing optimal search strategies for detecting clinically sound studies in MEDLINE.. H. 144-154. Faden & N. (in press). theme. 6. words. Qualitative researching.. Information graphics: A comprehensive illustrated reference. Retrieved from www. E. Beverly Hills. & Bleich. 51. (1981). Losee. (1998). P. B. Fitzpatrick. C. R. (1998). 1658-1659. 20(6). R. & Sinclair. When information retrieval measures agree about the relative quality of document rankings. J. Words. field content . M. Estabrooks. M.com Dixon-Woods. Evans.sagepub. & Kraft. V. (2001. Crooker. et al.

In J.. CA: Sage. D.. CA: Sage. (1997).. P.sagepub.com at UNIV OF TEXAS AUSTIN on November 17.). (2000). C. M.. New York: Bowker. J. Beverly Hills. 51. (2000). 2009 . Ulrich’s Periodicals Directory. Qualitative meta-analysis. Morse (Ed. Ennis. Wolf. Meta-analysis: Quantitative methods for research synthesis. & Emden. & Watkinson.. R. Downloaded from http://wjn. N. 311-326). Completing a qualitative project (pp. Thousand Oaks. F. S. (1986). & Stern. G. Crooks. Research in Nursing and Health. Qualitative metasynthesis: Issues and techniques. Docherty. 365-371. Empirical studies of end-user searching. Journal of the American Society for Information Science. A. S.178 Western Journal of Nursing Research Sandelowski. 1211-1231. Sutcliffe. M. 20. M. M. Schrieber. (1997)..