This action might not be possible to undo. Are you sure you want to continue?
Mapping Mixed Methods Research: Methods, Measures, and Meaning
J. Wheeldon Journal of Mixed Methods Research 2010 4: 87 DOI: 10.1177/1558689809358755 The online version of this article can be found at: http://mmr.sagepub.com/content/4/2/87
Additional services and information for Journal of Mixed Methods Research can be found at: Email Alerts: http://mmr.sagepub.com/cgi/alerts Subscriptions: http://mmr.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav Citations: http://mmr.sagepub.com/content/4/2/87.refs.html
Downloaded from mmr.sagepub.com at UNIV AUT CIUDAD JUAREZ on August 3, 2010
Toppenish. pragmatism. data collection. 2010 .com Downloaded from mmr.nav DOI: 10.com/journalsPermissions. this article details how the use of pre/post concept maps can be used to design qualitative interviews. p. 1999). 3240 Fort Road. its development is still ongoing.com Abstract This article explores how concept maps and mind maps can be used as data collection tools in mixed methods research to combine the clarity of quantitative counts with the nuance of qualitative reflections. 13). USA Email: jwheeldon@gmail. Based on the complexity of emergent research problems and the acknowledgment that multiple paths to meaning exist. 2004). relevant. options. 2008). WA. Keywords mixed methods measure. and approaches to consider. for a new generation of researchers the either/or approaches of the past must give way to more nuanced. abduction. researchers face what is best described as a series of choices (Palys.sagepub. In the past few years. As a more novel contribution. 2007. scholars in this journal and elsewhere have debated whether mixing multiple methods was possible and explored the theoretical basis for this integration. Another question that might be explored is whether and how data collection procedures from other disciplines can be used in 1 Heritage University.sagepub. and socially useful considerations (Rorty. concept maps. this approach offers an initial means to consider on what basis a mixed methods measure could be conceived. (Johnson & Onwuegbuzie. this article also explores how qualitative participant-generated mind maps can be used in multistage data collection to develop a mixed methods ‘‘salience score. Wheeldon. USA Corresponding Author: J. Wheeldon1 Journal of Mixed Methods Research 4(2) 87–102 ªThe Author(s) 2010 Reprints and permission: http://www. 1992). Toppenish. Based on more traditional mixed methods approaches. More recent lines of inquiry have focused on the construction of mixed approaches and how this may vary by discipline and the needs of individual researchers (Tashakkori & Creswell. mind maps Introduction The long-standing argument between qualitative and quantitative researchers has underscored broader political differences and research agendas (Jick. Once useful in creating insular research fiefdoms. sagepub. Although it appears to have emerged as the ‘‘third methodological movement’’ (Creswell & Plano Clark. Mixed methods research provides a variety of choices. Measures. Heritage University.Mapping Mixed Methods Research: Methods. WA 98948.’’ Although in many ways preliminary.com at UNIV AUT CIUDAD JUAREZ on August 3. and Meaning J.1177/1558689809358755 http://jmmr. 1979).
Mixed Methods Approaches: Theory and Practice As mixed methods research has grown in practice and recognition. Yoshioka. For mixed methods research. By focusing on solving practical problems. By introducing and differentiating concept maps. 2007). 2003). Greene and Caracelli (1997) have highlighted a number of purposes or justifications for mixing methods. the debate about the existence of objective ‘‘truth. Benjamin. p. perspectives. embedded design. In fields such as health. and meaning. 2001.88 Journal of Mixed Methods Research 4(2) mixed methods research and how these tools may influence and inform methods. 2003.com at UNIV AUT CIUDAD JUAREZ on August 3. 2007). and histories’’ of research participants (Ritchie & Lewis. maps can provide an accessible and innovative means to combine the clarity of quantitative counts with the nuance and perspective of qualitative reflections. these tentative explanations can be tested both Downloaded from mmr. the combination of quantitative and qualitative strategies has required new thinking about the theoretical basis for integrative research. The number and variety of research designs and approaches has required the consideration of an alternative epistemological framework.’’ or the value of subjective perceptions. 2006). Four major types of mixed methods designs have been presented. can be usefully sidestepped. this article provides specific examples of how concept and mind maps can be used. measures. Tashakkori & Teddlie. explanatory design. As such. Abduction is crucial in this respect. 3). & Shimomura. The mixed methods approach is perhaps of most interest to the ‘‘postparadigm’’ generation of scholars because it can produce more robust measures of association while explicitly valuing the depth of the ‘‘experiences. & Goodyear. Thus. and to show how the results from one method shape subsequent methods or research decisions. and intuition of researchers (Schurz. These include the triangulation design. 2007). Indeed. and exploratory design (Creswell & Plano Clark. one can both generate possible research solutions and at the same time attempt to integrate various theories and approaches (Tomiyamal. In this way. Through this iterative approach. Takeda. 2010 . These include the following: to test the consistency of findings obtained through different instruments. 2002). and pragmatism has emerged as a common alternative to the either/or choice of positivism and constructivism (Creswell & Plano Clark. 1998). to contribute to a broader discussion about data collection in social science research. experience. pragmatism allows for a more flexible abductive approach. This article is organized to present a short overview of existing approaches to mixed methods research and review how pragmatism and abductive reasoning are connected to the relationship between theory and data. One trend in social science research is the development of new means of data collection that are either more explicitly user-generated or that are less influenced by what may be sui generis research/participant interactions. education. to clarify and build on the results of one method with another. A variety of views on this paradigmatic issue have been suggested (Greene.sagepub. pragmatists have no problem with asserting both that there is a single ‘‘real world’’ and that all individuals have their own unique interpretations of that world (Morgan. maps appear flexible enough to be used both in established mixed methods approaches and in the construction of unique and novel mixed method measures. abductive reasoning allows for tentative explanations and hypotheses to emerge through the research process based on the expertise. alone and in combination. or inductive approaches that seek general conclusions based on specific premises. maps offer a unique way for research participants to represent their experiences while assisting researchers to make better sense of gathered data. Although existing models of mixed methods research exist and remain viable. instead of relying on deductive reasoning and general premises to reach specific conclusions. Through an innovative combination of existing knowledge. and engineering. participant-generated concept maps and mind maps are of resurgent interest (Nesbit & Adescope.
& London. These patterns might be better identified. 2003). 2003) and explored new ways to quantify qualitative research (Sandelowski. ‘‘vee diagrams’’ (Gowin. Nakkula. 2010 . or collect both qualitative and quantitative data at the same time (Gogolin & Swartz. & Bentall. sounds. 71). and even email and text messages (Creswell. Deductive Objectivity Generality Qualitative Approach Inductive Subjectivity Context 89 Pragmatic Approach Abductive Intersubjectivity Transferability theoretically and empirically.Wheeldon Table 1. Examples of various approaches include ‘‘box-and-whisker charts’’ and ‘‘stem-and-leaf diagrams’’ (Tukey. photographs. 2007). 2001). 1999). & Simpson. Jenkins. 1999. 2009). Despite these numerous examples. shared meaning. 2005). Stauber. 1974). Sells. 1992. & Huang. ‘‘knowledge mapping’’ (Dansereau. tools. p. Although many advance a broad definition of mixed methods that includes a wide variety of research methods and means of data collection. ‘‘mind maps’’ (Buzan. As Morgan (2007. One approach to the focus on intuition. It may be time to consider other forms of data collection that provide a means for researchers to see more easily participant meaning and use practical data analysis strategies through the flexibility abductive reasoning affords. or the review of various documents and audiovisual materials such as videotapes. Studies that employ mixed methods may do so in a variety of ways. Researchers may gather data through quantitative surveys and qualitative interviews (Baumann. researchers have usefully shown how to better explain quantitative findings with qualitative data (Aldridge. 1981). according to Ahlberg (2008). observations.sagepub. given the approach and types of data traditionally associated with quantitative and qualitative research. 1979). understanding based on shared meaning can benefit from the different approaches to research outlined in Table 1. and perception (Wheeldon. Key Issues in Social Science Research Quantitative Approach Connection of theory and data Relationship to research process Inference from data Source: Morgan (2007. In contrast. Yet additional approaches are needed to meaningfully embed and bring to light qualitative data within quantitative designs (Creswell & Plano Clark. and ‘‘spider diagrams’’ (Hanf. and understood through more graphic representations of knowledge. Fraser. visual representations in current research are generally limited to the presentation of data and explanation of findings. One reason may be that. visual depictions of data analysis have not become widely popular. 1977). recognized. Day. one can convert observations into theories and then assess those theories through action. it often appears this is an area in which many different types of graphic organizers are pitted against one another. Quantitative data are often based on instruments that measure individual performance and attitudes or use interview or observational data based on clearly predefined categories. 1994). Collecting different sorts of data may be challenging. Randall. Despite the wide variety of types. and transferability of data might be usefully connected to the idea that knowledge and understanding are themselves based on patterns (Kaplan. 1964). In mixed methods. p. and approaches since the 1970s. 2001). 71) suggests by moving ‘‘back and forth’’ between induction and deduction. J. A notable exception in this regard is Creswell Downloaded from mmr. too few are willing to acknowledge the shoulders of those on which they stand. more often traditional data collection means are employed in mixed method research. E. ‘‘concept maps’’ (Novak. Indeed. use different data procedures though multiple data collection phases (Rogers. In this way. experience. 1983). Past mixed methods approaches have sought to use qualitative data to develop quantitative data collection tools (Myers & Oetzel.com at UNIV AUT CIUDAD JUAREZ on August 3. 1981). qualitative data are generally based on themes that emerge through open-ended interviews. ‘‘clustering’’ (Rico. 1971). Way.
this approach was usefully expanded and developed in the seminal book Learning How to Learn (Novak & Gowin. and connected can serve as another means for researchers to collect data. and maps to visually present mixed methods approaches. 1994).90 Journal of Mixed Methods Research 4(2) Concept Maps connect are organized Related Ideas through different kinds of assists recall of Memories/ Experiences/ Perceptions/ Reflections hierarchically demonstrate are related to guide Links and Cross links Individual Understanding Meaning Making Figure 1. mind maps are more flexible tools in which a central governing concept is explored using groupings or areas. These are related to the way the maps are structured. These data can be used alone or to design additional data collection strategies. By asking participants to complete concept or mind maps. tools that graphically represent knowledge and experience may be especially useful. Of immediate interest is how concept maps and mind maps can be used to collect data in social science research. 1996). mapping knowledge is a technique that can demonstrate how people visualize relationships between various concepts (Lanzig. First presented formally by Novak (1981). 2010 . 1984) and the assumption that this meaning is based on the recognition of patterns (Miles & Huberman. and procedures. these maps and the ways in which concepts are presented. what they may represent. A concept map can be defined as a graphical tool for organizing and representing knowledge. perception.com at UNIV AUT CIUDAD JUAREZ on August 3. Concept Maps and Mind Maps in Social Science Research In general terms. to better explore and assess meaning. or memory. Yet important differences exist. and the ways in which their meaning might be interpreted.sagepub. Based on the notion that people attempt to make meaning of the world around them (Novak & Gowin. usually enclosed in circles or boxes. ordered. Downloaded from mmr. Lines and linking words between and among concepts suggest hierarchical relationships as presented in Figure 1. strategies. Whereas concept maps provide a more structured approach to explore connections between and among concepts using linking words. Both concept maps and mind maps focus on the meaning associated with visual representations of experience. knowledge. Concept maps include unique concepts. figures. Concept map of a concept map and Plano Clark’s (2007) mixed methods text that uses a variety of diagrams. 1984).
1996). VanEtten. Johnston. Yokoi. They can affect concentration and overall test performance by promoting interaction and engagement between the student and material (Hall & O’Donnell. cognitive. concept maps have most commonly been used as part of quantitative research designs. More recently. In education. & VanMeter. & McCleish. 2002.Wheeldon 91 Beliefs Examples Relationships Linkages Related Concept(s) Feelings CENTRAL THEME/ FOCUS Connections Personal Experience Examples Memories Figure 2. 2006). but in general. 1996). there is less specific research on mind maps than concept maps (Farrand.sagepub. they have been shown to be more effective to assist knowledge retention than attending class lectures. 1998). through selective searches of education and psychology databases. Constructing a mind map A mind map. or participating in class discussion (Poole & Davis. they have been used to facilitate knowledge integration and explore student understanding (Turns. 2005). Atman. they were further defined and developed over the next decade.com at UNIV AUT CIUDAD JUAREZ on August 3. or other items linked to and arranged around a central key word or idea. and evaluation (A. Mind maps have been used in nursing education (Rooda. Mueller. In the past. 2006. Figure 2 provides an example of how to construct a mind map. 1981). Mind maps are less formal and structurally more flexible. As is clear it begins with a central idea and builds outward. and mind maps as though they are interchangeable (Nesbit & Adescope. Freebern. Hussain. knowledge. Nesbit and Adescope (2006) discovered. tasks. in contrast. One reason may be that many appear to refer to concept. that more than 500 peer-reviewed articles have been published on the application of concept or knowledge maps and that most of these publications came after 1997 (Nesbit & Adescope. 2010 . & Bligh. 2000). 2006). & Hennessy. They may present ideas in a variety of ways (Buzan & Buzan. Bush. reading. 2000). This oversight belies the important differences Downloaded from mmr. Jenkins. ideas. is a diagram used to represent words. Interest in the use of maps in social science research is clearly increasing. 2006). First described by Buzan in 1974. & Adams. 2001). 1994). Maps may be an easier way to communicate one’s knowledge when compared with text writing (Czuchry & Dansereau. perhaps owing to their first use in science education (Novak. Pressley. care planning (Kern.
com at UNIV AUT CIUDAD JUAREZ on August 3. and/or compared to assess an individual’s understanding of a topic. A focus on structure remains an integral feature for many concept map researchers who insist that a number of steps are required for the creation of a ‘‘good’’ concept map. and the use of linking words to show how the various concepts are related (Novak & Caas. & Wolfe. 2004. To score a concept map. a hierarchical presentation in the map itself of general concepts on top and specific concepts on bottom. 2008). Rye & Rubba.sagepub. Shuman. These inherent differences provide some flexibility for researchers interested in using maps in different sorts of mixed methods research designs (Wheeldon. cross-links. Novak and Gowin (1984) describe the utility of maps to assess understanding in education. Gerchak. whereas mind maps usually are less formal and generally focus on one topic at a time. One way to think about the differences between concept maps and mind maps is to consider how they have been used in past research. Lyons. scored. Similar structural scoring systems have been Downloaded from mmr. Novak and Gowin (1984) suggested that maps be assessed by a subject matter expert based on the number of valid propositions. 2002). levels of hierarchy. How to score a concept map that exist and undermine their utility in research design. 2010 . Once created in this manner. and specific examples provided in the map. & Rowell. In short. and number of branchings. Concept Maps: Quantitative Research and Mixed Method Designs Concept maps are most commonly used in quantitative research. structured interview questions can be posed to the student to explore areas of student misunderstanding or confusion based on the student’s map. This may be because earlier versions of concept maps were used to explore science education (Stewart. concept maps often are more structured and include multiple concepts. and were often quantitatively scored by an expert to assess how understanding was demonstrated through the structure of the map itself. These include the generation of an initial ranked list of concepts.92 Journal of Mixed Methods Research 4(2) MAIN CONCEPT Links Links Scoring Links/Connectors – 1pt Examples – 2 pts Related Concepts – 5pts Cross links – 10 pts Related Concept connects Cross Link Related Concept connects connects connects example example example Figure 3. Figure 3 provides an example of how a concept map might be scored based on its structure. Van Kirk. various refinements have been presented (Besterfield-Sacre. 2009). Since then. 1979). maps can be assessed. They argue that by having students complete concept maps on certain topics.
These approaches score the quality of concepts and propositions within the maps (Rye & Rubba. Shavelson. experiences.sagepub. Other approaches rely on maps as a means to compare experts and learners (McGaghie. doubts persist about the combination of approaches based on what are very different foundational assumptions (Elliott. of additional interest was how and why these views changed over time. Through an open-ended survey and focus groups. exercises. 2008). 2008). The results were statistically significant. 2008). 2005).com at UNIV AUT CIUDAD JUAREZ on August 3. Forty-five students were assigned a unique identifier code and were asked to complete a map during the first pre-internship class and again during the last pre-internship class. some researchers have explored how to use concept maps as part qualitative research.Wheeldon 93 proposed based on the physical characteristics of the maps (Turns et al. 1995). This approach is in some ways similar to existing mixed method approaches that use quantitative surveys and qualitative interviews (Way et al. Kaya. In addition to scoring and comparing the pre/post maps. the pre/post maps were scored based on the number of individual concepts and on the complexity based on the number of hierarchies within the maps. and scenarios used to teach ethical decision making during the pre-internship course (Wheeldon.. 1985). specific interview questions can be designed to explore either of the pre/post maps as presented by the participant or the space and time in between the completion of the maps. To assess how views had changed over the course. 2010 . a research design of immediate interest uses concept maps as a pre/post data collection tool to quantitatively suggest how understanding. Mitchell. & Schultz. family. and/or perceptions can change over time (Kilic. Yet complications can arise when qualitative researchers adhere to the formal requirements of concept maps (Daley. Students not only identified more concepts. To explore student perceptions about these questions. This approach can usefully be applied to mixed method research. ethics. Heffernan & Stroup. Based on a predefined structure. 2001). and religion among others. and origins on average but also provided greater detail within their post maps about the connections between events. 2004). Within the literature disagreement exists about the means used to guide instruction on values and ethics (Cederblom & Spohn. Although not appropriate for all types of research. 2002) or identify different levels of concepts and subconcepts (Bayram. they provided their views on the importance of ethics to their placements and the value of the different approaches. the maps identified important individual values and ethics for each student along with their perceived origin such as school. However one chooses to score a map. Mind Maps: Qualitative Research and Mixed Methods Measures Despite the successes of mixed method researchers in demonstrating how different approaches to data collection and analysis can lead to a more vigorous interrogation of meaning. data were collected from students during the course of their 4-month criminal justice internship. the concept map simply served in place of another survey instrument. This may result in research that favors quantitative comparisons over unique participation representations. values. A variant of this approach was used in a pilot study to assess different teaching strategies for internship students related to values and ethics in criminal justice (Wheeldon. McCrimmon. 1991. As Wheeldon and Faubert (2009) note. In this case. More flexibility may be required if maps are to become a more commonly used data collection tool in social Downloaded from mmr. and the development of their ethical beliefs (Wheeldon. views. 2000) and develop scoring systems based on the convergence of concepts between a novice and expert map (Ruiz-Primo. & Thompson. Li. A useful example is Anne Raymond’s (1997) work in math education. 1994). This concern may also be relevant in a discussion of the utility of maps in social science research more broadly. 2000). By first comparing the pre/post maps of participants. & Dogan. this approach provides an innovative strategy for mixed methods based on pre/post data collection that may allow the researcher to further explore the graphic data within the maps through subsequent qualitative interviews. 2004)..
specific questions were asked of participants based on each participant’s map to clarify meaning. Instead of quantitatively scoring the maps. and legal technical assistance literature (Shaw & Dandurand. participants were asked to complete a mind map about their experience of a legal technical assistance project. the identification of themes within the maps led to another more complex research design that combined the frequency of individual variables identified over a variety data collection stages. these frequency measures can only provide a sense of whether and how often these variables were identified.sagepub. elements. 19 participants provided data over 1 year through a multistage research design. qualitative interviews were designed to explore the connections identified within the maps. or in summative and reflective statements. By combining the maps with the different stages of follow-up interviews. instead of focusing on how well the maps adhere to a set of predetermined rules. Finally. To analyze the interview data in a more meaningful way a mixed methods measure called a ‘‘salience score’’ was developed and tested. conclusionary and more reflective open-ended questions followed the more directive data collection stages. reflections. In the first step. and thus. in the third stage. unique. this study explored how participants viewed various components of capacity building and training during a 20-month justice reform project. capturing unique and individual perceptions. They also may be identified in one. researchers can consider how the construction of a participant’s map is connected to the meaning associated with a particular topic or experience. multiple. In that study (Wheeldon. 2009). free form. These initial calculations provide a sense of the number of times individual variables were identified in total across the data collection stages and the number of times each participant identified a variable across multiple data collection stages. These variables can then be quantified through the use of a concept counting system that records the frequency or presence of individual variables throughout data collection. the flexibility of mind maps may be more appropriate for qualitative research designs.com at UNIV AUT CIUDAD JUAREZ on August 3. through general or specific interviews. 2000). These general questions were open-ended and probed positive and negative experiences. Of some interest. it was hoped they would reflect on their experience as whole. in the fourth stage. and explore the map’s construction. An assumption of this study was that the way in which Downloaded from mmr. Based on the structure of each map. The construction of a mixed methods ‘‘salience score’’ may involve a number of separate yet rather simple operations. mind maps can be more unsolicited. 2006). This approach was used in a pilot study about how selected international justice trainers who worked abroad understood their role as experts (Wheeldon. gathered through a literature review. 2010 . 2007).94 Journal of Mixed Methods Research 4(2) science research (Wheeldon & Faubert. 2005). Individual variables might be identified in mind maps.’’ was developed to assess the effectiveness of various elements of training and capacity development in a legal technical assistance project in the Baltic State of Latvia. restating aspects of particular significance. or all stages of data collection. 2007). individual concepts. In the second stage of data collection. participants were asked more general interview questions. perceived results and challenges. 2009). In the first stage of data collection. as well as previously identified concepts. & Vernon. for qualitative researchers. and activities identified by participants in different stages of data collection can be recorded as variables. and user generated (Buzan & Buzan. or experiences is prized. Less structured and formal than traditional definitions of concept maps. Indeed. or provide additional clarifying commentary. probe connections. the ‘‘salience score. In addition to the general questions in the first and second stages. Based on emergent views within organizational and international development (Chodosh. the construction of a novel mixed methods measure. Watts. By providing participants an opportunity to identify areas not previously addressed. It has also been suggested that mind maps may provide a valuable means to collect more personalized and individualistic data from research participants (Tattersall. the frequency with which individual variables were identified through the multiple data collection stages was recorded. Thus. In total.
By explicitly privileging. a ‘‘salience score’’ or weighted measure was developed using a weighted count system (Stillwell. 2001) and then combined for each individual. User-generated concepts gathered through the maps might be deemed worth 4 points. 2000) and within the qualitative interviews (Sandelowski. open-ended. individual variables that emerged from user-generated.0 Source: Wheeldon (2009). common unique variables were identified in each mind map (Turns et al. Reference statement Total Salience score Frequency of Concept 0 1 0 1 2 5 Weighted Measure 4 3 1 2 Percentage 0 50. each participant’s individual variable salience score (IVSS) can be added together and the result divided by the total sample (n). Concepts identified by respondents based on specifically designed questions by the researcher could be worth the least at 1 point..Wheeldon Table 2. In this study. Tabulating the Salience Scoring Data Collection Stage 1.sagepub. and unsolicited data collection procedures can be treated as more valuable and given more weight in the overall measure. variables that emerge through more unsolicited data collection stages. a mixed method ‘‘salience score’’ can be tabulated for each individual variable in which both the frequency of the concepts identified and the way in which the data were collected can be combined. the salience score can incorporate both overall variable frequencies while accounting for variables identified throughout multiple data collection stages. given that participants came back to these concepts after several other data collection stages. By adding the points assigned through each stage of the data collection process. General interview 3. along with responses to general. the ‘‘salience score’’ as a measure is based on a quantification of qualitative data. it is based on the use of qualitative data to develop a quantitative data collection tool (Creswell & Plano Clark.com at UNIV AUT CIUDAD JUAREZ on August 3. By repeating this process.0 100 Valid Percentage 0 50. & John. Table 2 presents an example of how a salience score of 5 might be tabulated for a concept identified in two out of four stages of data collection. whereas conclusionary questions asked at the end of both the general and specific question sets could be worth 2 points. nonspecific questions worth 3 points. 1987). This operation can be represented by the formula OVSS ¼ ½ðIVSS 1 þ IVSS 2 þ Á Á Á þ IVSS n Þ=n: Although in some ways this approach is novel. For example. Specific interview 4. Mind map 2. through a weighted scoring scheme. When combined with the more nuanced qualitative data gathered through interviews. 2003).0 0 50. 2007).0 0 50.. This strategy allows the researcher to assign participants a score for each individual variable they identify depending on the stage(s) at which these variables were recorded. Winterfeldt. 2010 . salience scores for identified variables can produce values ranging from 0 (not salient) to 10 (extremely salient). this approach may provide a strengthened means Downloaded from mmr. For each variable identified in multiple stages of data collection. To get an overall variable salience score (OVSS) for the total sample.0 0 100. 2002). [AQ: 12 ] the variables were identified might more usefully demonstrate the relevance or legitimacy of a proposed association between an individual and a concept (Cash et al. instead of using qualitative findings to create and validate a quantitative data collection survey (Myers & Oetzel.0 100 95 Cumulative Percentage 0 50. To tabulate a score for each individual variable.
and context in the construction of meaning (Cassirer. Some participants have suggested that completing the map was ‘‘difficult. personalities. Keegan. the use of maps in social science research may require a more in-depth theoretical justification. By using mind maps. 1963). groups. maps provide an alternative to existing approaches such as those that rely on surveys and other more traditional means of data collection. Various map scoring systems exist and are still being developed (Ahlberg & Ahoranta. there is evidence that important differences between people. this may mean that surveys and interviews may focus on recording only one sort of data. the more weight should be assigned to the variables that emerge. and images. the variables in the maps may take on new significance. 2009). The pre/post mixed method concept map approach outlined above is perhaps a rather obvious example of how maps can fit within established mixed method approaches (Way et al. One problem with verbal interactions is that individual human experience often gets filtered and mediated by linguistic constructions (Whorf. & Ward. and learning styles can limit the utility of concept maps to gather data (Rohm. Less obvious is the approach taken to develop a ‘‘salience score’’ based on multistage data collection designs that include mind maps. It appears that whereas the graphic representation of experience may serve certain populations.. 2004). For data collection in social science research. 1997). 1994).com at UNIV AUT CIUDAD JUAREZ on August 3. The first relates to the observed resistance among some participants when asked to create maps. semantics. this justification might be built on the notion that individuals think more clearly and effectively if they can avoid the assumptions and inherent limitations built into language (Korzybski. methods. To ensure this significance is self-sustaining. 2010 . For many researchers concept maps need to be created based on strict criteria.96 Journal of Mixed Methods Research 4(2) to clarify and build on the results of one method with the perspective of another (Greene & Caracelli. By scoring pre/post maps and basing interview questions on the results. Indeed. it may alienate others. Mapping Mixed Methods: Discussion and Limitations Although the use of maps as data collection means provides a flexible approach for researchers and a novel experience for participants. the ‘‘salience score’’ ensures that variables identified in multiple stages of data collection emerge as central quantitative findings. important limitations exist. 1994). By reviewing the qualitative data that emerges from the interviews.’’ ‘‘strange. A core assumption here is that the more user-generated the data collection. because people learn in different ways and think using a combination of words. 1997). 1933). A related concern is how the maps themselves are constructed. it does appear as though maps provide a means of data collection that can be used as part of multiple mixed method designs. Although it seems clear that how people visualize relationships between various concepts or ideas can suggest more dynamic schemes of understanding (Mls. This can create a variety of practical complications for researchers and Downloaded from mmr. it is based on an assumption that maps and other graphic tools are a useful means to capture experience. Although more study is needed. 2008). graphics. Maps may provide a means to break out of conventional representations of experience through language and allow participants and researchers to access other kinds of information (Legard. Indeed. the focus on language as the sole means of communication may be arbitrary (Ausubel. 2003). 1946). This is important in two ways. Beyond the purview of this article. 2004). and measures in mixed methods research. this numeric finding can be explored and considered in more detail (Greene & Caracelli. 1956). Despite the potential for both of these approaches to contribute to new means.sagepub.’’ and ‘‘hard to complete’’ (Wheeldon. by definition more flexible and participant focused. based on the psycholinguistic assumptions about the role of syntax. and some suggest participants must be ‘‘taught’’ to create them properly (Novak & Caas.
Thus the maps and the responses to the general interview questions are treated as most important. although maps appear to offer a means to assist researchers to abductively infer meaning. to create a defined interface for participants. and flexibility. Derbentseva. 1995.’’ whereas validity refers to the degree to which the data in the map accurately reflect reality. such as CMap among others. researchers interested in using concept maps might investigate the use of computer software. This can better ensure consistency and facilitate concept quantification and map comparisons (Bayram. the ‘‘salience score’’ can be assumed to be a useful measure to assess the value of concepts that emerged through data collection. 2005).sagepub. 2). Conclusion As mixed methods research continues to grow and demonstrate its viability. reliability can be understood here to mean the degree to which the data in the map are ‘‘repeatable. combining. The suggestions provided in this article may offer a partial reply to some of these concerns. 2002). there are few—if any—studies that have attempted to weigh data in this way. Another challenge is specific to mind maps. maps may be especially difficult to read and/or interpret. Concept maps and mind maps may be important tools in this regard. Mixed methods research can no doubt continue to rely on traditional data collection means and ordering. One solution may be to limit maps to one page and provide clear instructions along with an exemplar map to participants when using mind maps to collect data (Wheeldon. Issues related to the quality of the data derived from maps nonetheless require more investigation. In addition. Although it may be tempting to argue that reliability and validity are addressed if data from participant maps also emerges through other sorts of data collection. If no significant differences are observed between demographic groups. & Caas. An additional concern relates to the salience score itself. Mixed methods may offer a useful platform for further and more in-depth exploration. integrity. this article has argued that more attention should be paid to how researchers collect. researchers can assess the statistical significance of demographic differences among groups in the sample using an independent t test (adjusted for multiple tests). p. One approach is to collect demographic data during data collection. As a means to quantify qualitative findings. the quality of the data within a map ought not be confused with the credibility of conclusions drawn from data gathered through multiple stages. this measure can be justified as providing a useful means to highlight the value of data collected through stages that are less directive and more user generated. 1979) and an ‘‘embarrassment of riches’’ (Chenail. 1995. Yet other models may exist. Downloaded from mmr.Wheeldon 97 participants related to the comparability of concept maps. A final consideration is related to the problem of validity and reliability for the data gathered from maps. analyze. the combination of individual scores without accounting for demographic differences within the sample is problematic. This article has explored how renewed interest in alternative forms of data collection may be based on a pragmatic understanding of intuitive and abductive connections between theory and data. To address this. and derive meaning from data in general. To be sustainable mixed methods requires a dialogue about which aspects of the work of researchers from different traditions can be practically used in combination. Safayeni. 2010 . Before combining the individual variable salience scores. By providing a visual record of understanding. Although defensible. Indeed. Whereas a general challenge in qualitative research is the difficulty in analyzing the data generated (Patton. or embedding findings through existent approaches.com at UNIV AUT CIUDAD JUAREZ on August 3. scholars would do well to reflect on a challenge posed by Morgan (2007). 2007). Identified by Trochim (1989). described variously as an ‘‘attractive nuisance’’ (Miles.
present associated methods and measures. I would also like to acknowledge the reviewers at JMMR whose comments and suggestion have significantly strengthened this article. September). and Stephen Easton of Simon Fraser University for their support and assistance. Sweden. The more flexible mind map might be best used in designs in which more open-ended qualitative data collection data are combined to tabulate a quantitative mixed method ‘‘salience score. & Ahoranta. The simple construction of the presented mixed measure ‘‘salience score’’ should allow it to be tested. 2010 . Ahlberg. it may be time for a renewed effort and examination of what sorts of data collection are best suited to research problems. Concept mapping as an innovation: Documents. revised. a weighted scoring systems may be a useful way to think about quantifying qualitative data. This article has argued that the more structured concept map may be better suited to designs in which quantitative scoring and comparisons of pre/post maps are used to shape and guide subsequent qualitative data collection. and perhaps validated. (2004. By combining data gathered through participant mind maps with other sorts of collected data. more study. this article has attempted to provide a theoretical justification for the use of concept maps and mind maps in mixed methods research. Pamplona.. yet it is far from certain that multiple poorly designed data collection strategies are always better. Paper presented at the First International Conference on Concept Mapping. Downloaded from mmr.’’ By explicitly valuing data collected through more unsolicited means. challenged. References Ahlberg. more of a focus on the choices researchers make in considering research problems may illicit valuable and unconsidered aspects of research and assumptions about the viability of various data collection tools.com at UNIV AUT CIUDAD JUAREZ on August 3. With this in mind. and consider some specific examples of how maps might be used as part of multistage data collection strategies. designs. and analyses. Tallin. Acknowledgments I would like to extend my thanks to Simon Verdun-Jones. September).98 Journal of Mixed Methods Research 4(2) they offer what might be seen as unique data that can be assessed both quantitatively and qualitatively. Although concept maps and mind maps offer a novel means of data collection. these approaches can combine the reliability of counts with the credibility of participant perception. Bill Glackman. Concept maps: Theory. (2008. M. The combination of these different research approaches and associated data analysis strategies may provide a means to gain a better understanding of phenomena under investigation. Funding The author received no financial support for the research and/or authorship of this article. Spain. methodology.sagepub. V. Declaration of Conflicting Interests The author declared no potential conflicts of interest with respect to the authorship and/or publication of this article. and research examples are needed to assess whether and to what extent maps can be usefully employed by mixed methods researchers. Estonia. memories and notes from Finland. reflection. technology. Estonia and Russia 1984-2008. As a methodological contribution. Indeed. M. Paper presented at the proceedings of the Third International Conference on Concept Mapping.
Greene & V. Downloaded from mmr. A. Dickson. J. Using narrative in social research: Qualitative and quantitative approaches. H.. and evaluating quantitative and qualitative research. P... D. Use of your head. No. B. Language and myth. CA: SAGE. V. J. Gowin. Cambridge. 85.. Caracelli (Eds. L. Benjamin. J. D. & Huang. Journal of Research in Science Teaching. (1995. Paper presented at the proceedings of the First International Conference on Concept Mapping. (2005). Hall.. 2010 . Contemporary Educational Psychology. J. F. W.). conducting. Upper Saddle River. Cash. J. & Hennessy. Unpublished master’s thesis. Evaluation.). Bayram. (2005). Hanf. J. technology.. N. (1981). (1995). & Jä. & Spohn. In G.. legitimacy and boundaries: Linking. J. (2002). Designing and conducting mixed methods research. 20(7). PA. Alcock. M. Nursing Standard. H. NY: Cornell University Press. 16. Chenail. 93. Eckley. The psychology of meaningful verbal learning. Mind mapping. 373-391. Lyons. W. Buzan. Evaluation of treatment for youth in the Drug Abuse Reporting Program. 74). J. B. Beschner & A. (1991). J. C. Greene. (2001). Advances in mixed-method evaluation: The challenges and benefits of integrating diverse paradigms (New Directions for Program Evaluation. T. methodology. Investigating classroom environments in Taiwan and Australia with multiple research methods. & Buzan. The Qualitative Report. 94-101. Retrieved December 11.. Gogolin. Thousand Oaks. (2007). (2005). The mind map book. B. J. 487-504. & Plano Clark. 23(2). M. R.. MA: Kennedy School of Government. D. Shuman.html Chodosh. The merits of mixing methods in evaluation. issues. & Caracelli. (1992). & Dansereau. 1-18.. London: SAGE. from http://www. (1971). H. J. (2005).com at UNIV AUT CIUDAD JUAREZ on August 3... C. 48-62. (1974). F.sagepub. T... Buzan. C. (1996). (1946). V. L. and treatments (pp. Presenting qualitative data. Adoptive fathers and birthfathers: A study of attitudes. F. 105-116.. Teaching of Psychology. Cognitive and affective outcomes of learning from knowledge maps. Ausubel. In J. D.. C. Youth drug abuse: Problems. Ithaca. Jenkins. J. NJ: Pearson. 14. (1979). L. M. Heffernan. 29. N. Baumann. New York: New York University Press. 2009. & Goodyear. Journal of Engineering Education. The effectiveness of concept and software mapping for representing student data and process schema in science. A quantitative and qualitative inquiry into the attitudes toward science of nonscience college students. University of Pittsburg. A model for teaching criminal justice ethics. E. 7(1). 93(1). Scoring concept maps: An integrated rubric for assessing engineering education. Defining and describing the paradigm issue in mixed-method evaluation. (2004).. Fraser. Gerchak. (1999). Elliott. (2002). A. B. Greene. J. December). M. San Francisco: Jossey-Bass.. C. 225-230. Journal of Educational Research.edu/ssss/QR/QR2-3/presenting. W. T. R. Czuchry. Creswell. (1996). Educating.. Besterfield-Sacre. & Wolfe.ger. Node-link mapping as an alternative to traditional writing assignments in undergraduate psychology courses. B. D. B. September). F. 25-44.Wheeldon 99 Aldridge. Lexington. S. Credibility.nova.. 2(3).. Mapping: A technique for translating reading into thinking. (1963). Journal of Criminal Justice Education. Clark.. Educational research: Planning. W. 36(5). Hussain. Cassirer. Journal of Reading. Medical Education. (1985). Police ethics: Hard choices in law enforcement. London: BBC Books. S. T. D. & Swartz. Global justice reform: A comparative methodology. 91-96. New York: Grune & Stratton. (2004.. Spain. Pamplona. & O’Donnell. Concept maps: Theory. Daley. Farrand. M. Friedman (Eds. Sells. L. J. (1997). New York: John Jay Press. 21. MA: Lexington Books. London: BBC Books. assessment and decision making. 26-31. The efficacy of the ‘‘mind map’’ study technique.. Cederblom. (2000). & Simpson. C. 2. Creswell. 201-218. Child & Adolescent Social Work Journal. & Stroup. D. S. E.. C. (1999). L. New York: Harper & Brothers. Dansereau. 571-628).
San Francisco: Chandler Press. 436-455. Z. C. Ontario: Thompson Canada.and post-laboratory concept maps on students’ attitudes toward chemistry laboratory in university general chemistry. (2001). & Onwuegbuzie. Rural adolescent perceptions of alcohol and other drug resistance. D. O. J. Hacker. Thousand Oaks. Mixing qualitative and quantitative methods: Triangulation in action. 347-367). K. A. McCrimmon.. P. R. & Huberman. (2003). Korzybski. From concept mapping to qualitative modeling in cognitive research. A. M.. The metacognition of college studentship: A grounded theory approach. Freebern. 45. B. J. Raymond.. Ritchie & J.pdf Morgan. J. CA: SAGE. T. (2003). M. 1-28. Advanced Physiological Education.com at UNIV AUT CIUDAD JUAREZ on August 3.. Pensacola: IHMC Florida Institute for Human and Machine Cognition. Myers. Mind-mapped care plans: A remarkable alternative to traditional nursing care plans. Administrative Science Quarterly. O. A. J. (2006). Journal of Nursing Education. (1979).nl/artikel1/ Legard. 138-169). G. M.cmc. 51. Mixed methods research: A research paradigm whose time has come.to. Review of Educational Research. M.. 2006.. Effects of students’ pre. Communication Quarterly. (1997). R. Johnston..100 Journal of Mixed Methods Research 4(2) Jenkins. Kaplan. 72. J. Downloaded from mmr.. 19(3). Johnson. 14-26. & Oetzel. & Davis.. 33(7).. (2006). Qualitative data as an attractive nuisance: The problem of analysis. Research decisions: Quantitative and qualitative perspectives (3rd ed. Quantitative concept mapping in pulmonary physiology: Comparison of student and faculty knowledge structures. D. L. 590-600. J. M. M. A.. Paradigms lost and pragmatism regained: Methodological implications of combining qualitative and quantitative methods. NJ: Erlbaum. Qualitative evaluation and research methods. from www. (2004). UK: Cambridge University Press. Journal of Mixed Methods Research.. Patton. VanEtten. 413-448.. The conduct of inquiry: Methodology for behavioral science. Child Study Journal. (2007). (1984). J. 24. N. In D. Exploring the dimensions of organizational assimilation: Creating and validating a communication measure. A. J. Mind-mapped care plans: Integrating an innovative educational tool as an alternative to traditional care plans. A. J. & A. G. Focus on Learning Problems in Mathematics. K. The theory underlying concept maps and how to construct and use them (Tech. Concept mapping to measure outcomes in a study abroad program. & Bligh. 25(1). K. (1996).utwente. (1994). S. Lanzig. Novak. D. Palys. . Kilic. B. 12-20.). Retrieved August 19. (1981).sagepub. 43(1). (2001). & Thompson. Pressley. Everything you always wanted to know about . 23(1). D. W. Dunlosky. The American Biology Teacher. 76. Learning with concept and knowledge maps: A meta-analysis. Mahwah. Thousand Oaks. Keegan. L. Retrieved October 16. CA: SAGE. J. Graesser (Eds. TX: Institute of General Semantics. concept mapping. Science and sanity: An introduction to non-Aristotelian systems and general semantics. . (1979). K. 211-224. 24. The use of concept mapping in qualitative research: A multiple case study in mathematics education. (1998). Istanbul.. & Adescope. 26(2). 112-119..).. D. O. Social Work Education.) Newbury Park. (2006). Educational Researcher. T. 2008. Mueller. & VanMeter.. T. (1933). 48-76.). C. B. Novak. D. Paper presented at the 18th International Conference on Chemical Education. J. Fort Worth. Bush. D. Miles. J. M. D. In J. CA: SAGE. (1992). from http://utto1031. Cambridge. Metacognition in theory and practice (pp. L. (2004).. A. (2004).ihmc. M.. Q. W. Learning how to learn... Novak. J. Kern. Jick. A. Nurse Educator. (1964). 1(1). Poole. C. (2002). Mitchell. (2008). 602-611. Turkey. Kaya. Lewis (Eds. Applying learning psychology and philosophy of science to biology teaching. & Dogan. J. & Caas. Qualitative research practice: A guide for social research students and researchers (pp. Administrative Science Quarterly. McGaghie. 75-80. E. 31. Toronto. Nesbit. Miles. & Gowin. (2000).). In-depth interviews.. S. B. A. 61-77. Qualitative data analysis: An expanded sourcebook (2nd ed. & Ward. 2010 .us/papers/cmc2004-159. Rep. Mls. Yokoi. & McCleish.
W. A. Rogers. S. Qualitative research practice: A guide for social science students and researchers. J. Concept maps: A tool for use in biology teaching. A. No. (1994). Way. 12. Atlanta. 230-240. Wheeldon. (1998). Watts.. H. J. Turns. J. (2003). Journal of Mixed Methods Research. A. (1994) Effects of mind mapping on student achievement in a nursing research course. British Columbia. & Schultz. Mapping ethics. Dusseldorf. GA: Personality Insights. Mind mapping as a tool in qualitative research. M.. Rye. 43. Models of abductive reasoning (TPD Preprints Annual 2002 No. J. 25-27. 87-110. R. Trochim. Stewart. values and problem solving among criminal justice students. J. 2(1). N. Mixed methodology combining quantitative and qualitative approaches. (2006). ˜ Safayeni. R. F. Menlo Park. L. (1989). (2003). Schurz & M. Tomiyamal.com at UNIV AUT CIUDAD JUAREZ on August 3. 171-175. Germany: University of Dusseldorf. Concept maps for engineering education: A cognitively motivated tool supporting varied assessment functions. R. A Theoretical Note on Concept Maps and the Need for Cyclic Concept Maps.. J. Atman. IEEE Transactions on Education. M. Tashakkori. Management Science. (2008. Social Psychiatry and Psychiatric Epidemiology. Derbentseva. T. Tashakkori. 7. 24. G.Wheeldon 101 Rico. (1983). Tattersall. L.. (2009).. (2003). M. Ritchie. Rohm. J.sagepub. Nurse Educator. 38. 42(7). Randall. MA: Addison-Wesley. Simon Fraser University. Downloaded from mmr. 32-33.. Comparing hierarchical and nonhierarchical weighting methods for eliciting multiattribute value models.. Rorty. 41. St. G. Shavelson.. 720-727. E. 2010 . (1999). Positive personality profiles. M.. 49).. Reading. British Columbia.. V. J. D. (Eds..). (Eds. 33-45. R. P. Wheeldon.. 164-173. Writing the natural way: Using right-brain techniques to release your expressive powers. Takeda. J. C. S. Van Kirk. Philosophy and social hope. Rooda. 3-6. Effective technical assistance in crime prevention and criminal justice. (2002). Tukey. F.. (2007). (2001). CA: SAGE. Winterfeldt. & Vernon. A. J. P. Helsinki. & Rubba. 99-141.. 103. Research in Nursing & Health. Real qualitative researchers do not count: The use of numbers in qualitative research. R.. Sandelowski. C. J. Y. (1979). J. J. (2000). & Dandurand. & London. Mixed methodology across disciplines. J. W. 1). Nakkula. Thousand Oaks. (2008). S. Burnaby. & Canas. Philosophical prepublication series of the Chair of Theoretical Philosophy.. R. 331-357. 102(1).. Thousand Oaks. MO. (2005). H. Stauber. November). & Shimomura. Y. Concept mapping: Soft science or hard art? Evaluation and Program Planning. 741-766. (1977). 23. J. Nursing Times. Paper presented at the American Society of Criminology Annual Meeting. & John. A. (2001). & Rowell. W. Abduction for creative design (AAAI Tech. Li. A. Canada: School of Criminology. R. (2007). Schurz.. M. Bringing British Columbia to Latvia: Canadians reflect on expert designation in an international criminal justice project. Wheeldon. (2002). E. London: Penguin. Stillwell. Journal of Research in Science Teaching. Mapping international knowledge transfer: Latvian-Canadian cooperation in justice reform.. Unpublished doctoral dissertation. & Teddlie. A patient’s understanding and participation in a trial designed to improve the management of anti-psychotic medication. 442-450. On the validity of cognitive interpretations of scores from alternative concept-mapping techniques. Burnaby. A.. 19(6). School Science & Mathematics. C. In G. & Bentall.. N. & Creswell. Depression and substance use in two divergent high school cultures: A quantitative and qualitative analysis. Rep. Werning (Eds.. Louis. SS-03-02).). Ruiz-Primo. & Lewis..).. CA: SAGE. & Adams. M. Educational Assessment. (1994). Exploratory data analysis. Day. W. Scoring concept maps: An expert map-based scheme weighted for relationships. Finland: HEUNI. P. Journal of Youth and Adolescence.. ¨ ¨ Shaw. Y. Ca: AAAI Press.. Yoshioka. Simon Fraser University. (1987). 33. New York: Penguin Putnam. The American Biology Teacher. Canada. Maximizing the effectiveness of the technical assistance provided in the fields of crime prevention and criminal justice (HEUNI Report Series No.
International Journal of Qualitative Methods. Language.com at UNIV AUT CIUDAD JUAREZ on August 3. B.). (1956). Cambridge: MIT Press.102 Journal of Mixed Methods Research 4(2) Wheeldon.sagepub. and data collection in qualitative research. thought and reality: Selected writings of Benjamin Lee Whorf (J. L. J. B. (2009). Whorf. & Faubert. J. Downloaded from mmr. Framing experience: Concept maps. Carroll. mind maps. 8(3). Ed. 68-83. 2010 ..
This action might not be possible to undo. Are you sure you want to continue?
We've moved you to where you read on your other device.
Get the full title to continue listening from where you left off, or restart the preview.