Critical Thinking in Graduate Medical Education: A Role for Concept Mapping Assessment?

Daniel C. West; J. Richard Pomeroy; Jeanny K. Park; et al.
Online article and related content current as of November 10, 2008. JAMA. 2000;284(9):1105-1110 (doi:10.1001/jama.284.9.1105) http://jama.ama-assn.org/cgi/content/full/284/9/1105

Correction Citations Topic collections Related Articles published in the same issue

Contact me if this article is corrected. This article has been cited 20 times. Contact me when this article is cited.

Medical Practice; Medical Education
Contact me when new articles are published in these topic areas.

September 6, 2000
JAMA. 2000;284(9):1175.

Subscribe
http://jama.com/subscribe

Email Alerts
http://jamaarchives.com/alerts

Permissions
permissions@ama-assn.org http://pubs.ama-assn.org/misc/permissions.dtl

Reprints/E-prints
reprints@ama-assn.org

Downloaded from www.jama.com at Penn State Milton S Hershey Med Ctr on November 10, 2008

Richard Pomeroy. West.51-0.jama.and thirdyear residents compared with first-year residents.and third-year residents) from a university-based pediatric training program underwent concept map training.16). Interrater correlation of map scoring ranged from weak to moderate for the preinstruction map (r= 0.and third-year residents scored significantly higher than first-year residents before instruction (mean [SD] score of 472 [116] vs 371 [102]. tools to evaluate the quality of students’ conceptual frameworks are limited because of the difficulty in designing and administering objective measures. West.10-0. Results Total CMA scores increased after instruction from a mean (SD) preinstruction map score of 429 (119) to a mean postinstruction map score of 516 (196) (P=.04). 1105 ©2000 American Medical Association. The study was conducted in May and June 1999. University of California. MD. whether concept maps can be scored reliably. 561 [203] vs 456 [179]. completed an education course on seizures. respectively.01) than first-year residents. 2008 . All rights reserved.284:1105-1110 www. and interrater correlation of map scores. Park.03). despite their critical importance to physicians’ evolving clinical expertise. The CMA score had a weak to no correlation with the American Board of Pediatrics In-training Examination score (r=0.2-9 This evolving conceptual framework supports expertise in critical thinking and clinical problem solving. Setting. and how well CMA scores relate to the results of standard in-training examination.edu). respectively.jama. P=. Concept mapping is a technique developed by Jo- Context Tools to assess the evolving conceptual framework of physicians-intraining are limited.10-12 Despite its critical importance in the education of future physicians. Conclusions Our data provide preliminary evidence that concept mapping assessment reflects expected differences and change in the conceptual framework of resident physicians. 2000—Vol 284. MD Jonathan Sandoval. Davis. Main Outcome Measures Preinstruction map total scores and subscores in 4 categories compared with postinstruction map scores. Maps were scored independently by 3 raters using a standardized method. 2516 Stockton Blvd. School of Medicine (Drs West. map scores of second. Corresponding Author and Reprints: Daniel C. In this article. Park. MD J. JAMA. MD Elise A. Many medical schools have adopted alternative teaching and learning strategies that challenge students to become self-directed learners who continuously expand the limits of their knowledge and seek evidence for clinical decision making. PhD Jeanny K. Design. University of California. September 6.ORIGINAL CONTRIBUTION Critical Thinking in Graduate Medical Education A Role for Concept Mapping Assessment? Daniel C. but not after instruction (mean [SD] scores. Davis. 9 Downloaded from www.13-15 Unlike most conventional ex- Author Affiliations: Department of Pediatrics. Concept mapping assessment and standardized testing may measure different cognitive domains. and Participants A group of 21 resident physicians (9 first-year and 12 second. Sacramento.com D seph Novak in the 1970s in which students create a 2-dimensional diagram outlining their understanding of the relationships between and among important concepts within a given subject area.com at Penn State Milton S Hershey Med Ctr on November 10. we describe an application of concept mapping assessment (CMA) that provides a “window” into learners’ organization of knowledge at various points in training. (Reprinted) JAMA. Second. as opposed to rote memorization of facts or procedural algorithms. No. P= . 2000.74-0. Objective To assess whether CMA reflects expected differences and changes in the conceptual framework of resident physicians. drew a preinstruction concept map about seizures. Department of Pediatrics.1 The root of these ideas lies in the constructivist theory of learning. and Gerstenberger) and Division of Education (Drs Pomeroy and Sandoval). and then drew a postinstruction map. Second.88). which states that meaningful learning occurs when students assimilate new knowledge into a preexisting conceptual framework.54). CA 95817 (e-mail: dcwest@ucdavis. PhD URING THE PAST FEW DE cades medical curricula have increasingly reflected the belief that students should understand concepts. Concept mapping assessment (CMA) enables teachers to view students’ organization of their knowledge at various points in training.and third-year residents had greater preinstruction map complexity as measured by cross-link score (P=.69) and moderate to strong for the postinstruction map (r= 0. Gerstenberger.

A total of 21 residents enrolled: 9 first-year residents. Example Concept Map Water changes needed by made of year residents. concept links. In a manner similar to propositions. All study subjects participated in a standardized concept map training exercise based on previously described methods.14 1106 JAMA. crosslinks are labeled with a phrase that describes the relationship of the domains. The linking of a series of related concept links in a hierarchical fashion represents a domain of knowledge. 9 (Reprinted) Maps were scored independently by 3 different raters.16-18 Therefore. which was approved by our institutional human subjects committee. and across domains (cross-links).14 Within 2 to 3 weeks of completing map training. Three levels of hierarchy are represented in Figure 1. Creating a concept map involves connecting related concepts using arrow lines with a statement or proposition written above the line describing how the concepts are related. as a learner’s conceptual framework changes. Each participant underwent concept map training followed by drawing a preinstruction concept map about the topic seizures. Examples are labeled and connected to the related concept in the most subordinate position on the map. They then participated in a 3-session seizure education course followed by drawing a postinstruction map about seizures. All rights reserved. we undertook this preliminary study to assess whether CMA can measure predicted differences and expected changes in the conceptual framework of resident physicians.3. primarily because there is limited information regarding its validity and reliability. and 4 third-year residents. Theoretically. Concept Maps We used a modification of the “hierarchical technique” of concept mapping in which a concept map consists of 5 HIERARCHY States LEVEL 1 Living Things Molecules determines (cross-link) are in can be can be can be Solid Liquid Gas LEVEL 2 is a type of is a type of Motion as in as in as in as in Plants example Lake example Steam LEVEL 3 oak tree Animals example Snow Ice Lake Tahoe components: concepts.14. CMA may provide unique insights into how an individual organizes his or her knowledge or comes to think about a problem. cross-links. One week after completing the course. In the example map. Downloaded from www. when depicted on a map. and examples (see FIGURE 1 for an example concept map of water). draw their concept of seizures on a blank sheet of paper (preinstruction map). and without study aids.14.CONCEPT MAPPING ASSESSMENT aminations. In Figure 1. especially in high school and college science education. we sought to determine whether concept maps could be scored in a reliable way.12. enclosed with a circle.20. Reproduced with permission from Cambridge University Press. Lake Tahoe is a specific example of a lake. participants were given 30 minutes to independently. blinded to the identity ©2000 American Medical Association.jama. METHODS Study Subjects and Design All pediatric residents (33 total) at our institution were eligible to participate in this study.19 A concept is defined as a perceived regularity in events or objects designated by a label and. Note the cross-link (bold arrow) between the concepts motion and states. represents the foundational unit of a concept map. The expression of the hierarchy between concepts is indicated by the direction of the arrow in the concept link with the most general concepts arranged at the top and more specific concepts arranged hierarchically below. 2000—Vol 284.21 In Figure 1. hierarchies. he or she links new propositions sequentially (concept links). general to specific (hierarchy). motion connected to states by the proposition determines defines a cross-link. thus demonstrating integration of knowledge. Residents then participated in a 3-session (1 hour per session) seizure education course consisting of case-based instruction. A concept linked to another concept by an arrow line with a propositional statement. it has only rarely been used as an assessment method.22 Scoring Method my dog Example map of water demonstrating the basic components of hierarchical concept maps. Cross-links are the method by which the map author indicates how different domains of knowledge are related to each other and are a critical measure of map complexity. In addition.com at Penn State Milton S Hershey Med Ctr on November 10. No. 2008 . While concept mapping has been used frequently as a learning tool. called a concept link. residents drew a postinstruction concept map about seizures under the same conditions as before. the concept water connected to living things by the proposition needed by represents a concept link. Written informed consent was obtained. 8 secondFigure 1. September 6.

hierarchy.03). To assess the relationship of concept map scores to standard written examinations. while FIGURE 3 shows a highscoring map from a resident in the second.7.and secondyear residents took the examination within 1 month of completing this F IGURE 2 shows a complete lowscoring concept map from a first-year resident. (Reprinted) JAMA.and thirdyear residents (n=12) into 1 group and compared their scores with those of the first-year (n = 9) residents using the 2-sample Wilcoxon rank sum test. Downloaded from www.jama. Maps were scored in 4 categories: concept links. September 6. these differences did not reach significance. Concept Map Scores After Educational Intervention Total scores increased significantly after the educational intervention (P=. resulting in a low score. The postinstruction maps demonstrated that while the total and subscores for second.15. cross-link (P=.04) and cross-link scores on the preinstruction map (P=. The value of each scoring category represents the depth of thinking required to form the category. 2000—Vol 284. the seizure education course. cross-links. and examples) were generated for each map. crosslinks. Of particular note is that the significant difference in total score and cross-link score seen on the preinstruction maps was no longer apparent after instruction (P =.02) and concept link (P=. The correlation of each rater’s total scores and subscores for each map was tested using Spearman rank correlation. Low-Scoring Concept Map Seizures includes many is treated with year residents had significantly higher total (P =. No. RESULTS All 21 participants completed all phases of the study including concept map training. and example scores based on level of training.and third-year residents to firstyear residents using the 2-sample Wilcoxon rank sum test. 9 1107 ©2000 American Medical Association. and both preinstruction and postinstruction concept maps. First. with numerous crosslinks in the map from the second. we compared the most recent American Board of Pediatrics Intraining Examination (ITE) scores of second. TABLE 1 summarizes the mean standard total and mean standard subscores of study participants by level of training. and examples. To assess whether there was a difference in map scores based on level of training or experience. We compared preinstruction map to postinstruction map scores using the Wilcoxon signed rank test.01) subscores also increased significantly.com at Penn State Milton S Hershey Med Ctr on November 10.and third-year residents remained higher. Preinstruction and postinstruction maps from 1 subject are has numerous Types Medications includes includes including Etiologies includes includes Secondary includes Primary Phenytoin including including Vascular includes Trauma including includes includes Phenobarbital Absence includes includes includes Carbamazepine Stroke Infections includes Myoclonic Jerks includes Valproic Acid Electrolyte Disturbances example example includes Meningitis Tonic-Clonic Complicated Febrile low sodium low glucose Encephalitis Common Febrile Reproduction of a hand-written concept map of seizures by a first-year resident. Subscores for hierarchy and examples both decreased slightly. but this change did not reach significance. A total score and subscore for each category (concept links. respectively). Note the complexity.CONCEPT MAPPING ASSESSMENT of the map author. were assigned to each map. and using a modification of the structural scoring method previously described. hierarchy. Tex).01) compared with first-year residents.and third-year group compared with the linear simplicity of the first-year resident’s map.0 (College Station. and each level of hierarchy was given 5 points. respectively. while third-year residents took the examination 10 months prior to completing the study. All rights reserved. Note the absence of crosslinks and the consistent use of 2 to 3 levels of hierarchy. Concept Map Scores Based on Level of Training Statistical analysis was done using STATA. 2008 . version 6. All other data analyses were done using standard total scores and standard subscores.14 Each valid concept link was given 2 points.and third-year group. A cross-link was given 10 points.14 Each rater was a board-certified pediatrician who underwent a 30-minute concept map scoring training exercise immediately prior to grading the maps. The second. In addition. we combined the more experienced second. hierarchy. We then tested the relationship of concept map scores to ITE scores using Spearman rank correlation. There was no significant difference between concept link.16 and . Invalid links or concepts were given 0 points.and thirdFigure 2.7. Standard total scores and standard subscores consisting of the sum of individual rater total scores and subscores. Data Analysis study. while an example counted for 1 point.

The correlation between ITE scores and postinstruction map total scores. 2000—Vol 284.88). The vague concept cerebral irritation included as a focal point in the preinstruction map was omitted in the postinstruction map. cross-link (P=. 0.16 . TABLE 2 summarizes the mean standard score and subscores of all subjects before (preinstruction map) and after (postinstruction map) the seizure education course. No. Total scores increased significantly after the educational intervention (P =.75 Preinstruction Map 472 (116) 371 (102) 161 (87) 71 (46) 182 (83) 173 (58) 68 (25) 59 (11) 64 (19) 67 (27) Postinstruction Map 561 (203) 456 (179) 215 (136) 131 (111) 233 (78) 211 (78) 62 (20) 61 (25) 59 (19) 54 (27) sure.02) and concept link (P = . resulting in a higher map score. Interrater Correlation of Concept Map Scores *Data are presented as mean (SD) standard scores.53.86 . 243 (94).15 .03). High-Scoring Concept Map Seizures can be caused by caused by caused by can be usually treated with Medications can be can be Epilepsy requires caused by Fever due to Generalized sometimes treated with type of Diet Lorazepam Phenobarbital can cause can cause example Education as in prevention of as in as in Infection Abnormal Glucose can be as in can be Respiratory leads to ketogenic diet Toxicity Medications Trauma Low Glucose due to Increased Intracranial Pressure can cause type of Neurologic type of Hypoxia type of Respiratory Depression 911 Call Nonaccidental type of as in Brain Abscess caused by caused by caused by Encephalitis type of caused by can result in due to Shaken Baby caused by Fall Accidental Ingestion High Glucose due to Meningitis caused by caused by caused by Virus spread by prevented by Insects spread by spread by example Humans prevented by can result in Neuronal Shear causes leads to requires causes as in as in requires as in often implicated Poor Intake Fungus Cerebral Edema results in Diabetes can be Tumor can be Bacteria Acute Hypertension treated with Intracranial Bleed detected by Motor Vehicle Accident Prevention required by Alcohol Bike Accident Primary as in Metastatic as in mosquito Osmotic Diuretic example Astrocytoma Breast Cancer Immunizations NeuroImaging mannitol as in Law as in Helmets Seat Belts Reproduction of a hand-written concept map of seizures by a resident in the second.05). The resident added a new conceptual domain for prevention and education as well as new crosslinks to increased intracranial pres- The mean (SD) ITE score of secondand third-year residents was 358 (135) and of first-year residents.34.51-0.01 .01) subscores also increased significantly.54). and 0.29. Note the complexity of the map as manifested by frequent cross-links and the consistent use of 5 to 7 levels of hierarchy.04 . All rights reserved. preinstruction map cross-link scores. P = .CONCEPT MAPPING ASSESSMENT Figure 3. The interrater correlation for subscores demonstrated an almost identical range to that seen in the analysis of total scores.54. The interrater correlation was weak to moderate (r=0.and Third-Year vs First-Year Residents* Secondand Third-Year Residents Component Total Cross-link Concept link Hierarchy Examples Total Cross-link Concept link Hierarchy Examples First-Year Residents P Value † . The strongest correlation was between ITE score and the example score on the preinstruction (r =0. Comparison of Total Concept Map and Subscores in Second.69) for the preinstruction map and moderate to strong for the postinstruction map (r =0. In addition. September 6. P=.01) and postinstruction map total score (r = 0. but this change did not reach significance.10-0.74-0.and third-year residents were significantly higher than those of first-year residents (P = . Table 1. Concept Map Scores and ITE There was weak to no correlation between CMA scores (total and subscores) and ITE scores (r=0. and postinstruction map cross-link scores was r=0. respectively. The ITE scores of second.24 .jama. †For comparison by 2-sample Wilcoxon rank sum test.com at Penn State Milton S Hershey Med Ctr on November 10.01).74 . Subscores for hierarchy and examples both decreased slightly.59 .and third-year group. Downloaded from www. 1108 JAMA. 2008 .20. shown in FIGURE 4.80 . See “Scoring Method” subsection for explanation of scoring method. 9 (Reprinted) ©2000 American Medical Association.

more experienced residents appeared to organize their knowledge differently. rater training would be important to any future applications. expressing links between concepts that were not observed by less experienced residents.06 *Data are presented as mean (SD) standard scores. notably.jama.16 While there may be other explanations. This finding is consistent with other reports.CONCEPT MAPPING ASSESSMENT COMMENT We found that concept map scores of residents increased significantly following completion of an educational program after which conceptual framework would be expected to change.03 Cross-link 123 (85) 179 (130) . All rights reserved. tion suggests that CMA measures a different knowledge characteristic than do multiple choice examinations. We also found that while ITE scores of second. This level of scoring reliability is similar to that reported by others and suggests that gains in scoring experience lead to increased consistency across raters. †For comparison by Wilcoxon signed rank test. Partial Reproduction of Preinstruction and Postinstruction Concept Maps From the Same Subject Preinstruction Map Seizures treated by may be may be Focal caused by caused by Antiepileptic drugs example Generalized caused by Dilantin example example Phenobarbital Tegretol Cancer can be Cerebral Space Irritation Occupying Masses caused by Meningitis caused by leading to Infection leading to can be Toxins like Encephalitis caused by caused by caused by Infectious Lead Bacteria Viral Postinstruction Map Seizures can be caused by can be caused by can be caused by as prevention of as in Trauma as in Fever can be caused by Education can be caused by like Nonaccidental Toxin Accidental as in as in Abnormal Glucose which leads to Increased Intracranial Pressure can be Infection type of as in as in Respiratory Shaken Baby causing causing causing Neurologic leads to Blood as in Bike Accident can be as in as in due to Neuronal Shear leading to Intracranial Bleed requires Fall High Low due to as in as in Encephalitis which leads to requires Cerebral Edema Motor Vehicle Accident often implicated in Ingestion as in Brain Abscess Meningitis Diabetes Prevention Alcohol Reproduction of approximately 50% of both a preinstuction and postinstruction concept map from the same subject. these findings provide important preliminary evidence supporting our hypothesis that CMA is a valid measure of conceptual framework and the expected change in that framework with new learning. we also found that CMA scores of residents with more training and expertise were significantly higher than those with less training. there is clear evidence that concept mapping can be an effective learning tool in a variety of classroom settings— including the education of health care 1109 ©2000 American Medical Association. While not conclusive. after all residents participated in the same educational intervention. (Reprinted) JAMA. such as final course grades or scores on standardized tests. 2008 . as one might anticipate. and a change in cross-linking between concepts. Figure 4). which demonstrates the incorporation of additional concepts into the postintervention map.com at Penn State Milton S Hershey Med Ctr on November 10. was greater on postinstruction maps. these changes are reflected in a statistically significant increase in total map score and concept link and cross-link subscores. Thus. This difference was the result of greater map complexity. 9 Downloaded from www. which note that CMA scores of college students do not correlate with conventional measures of learning. ture. The issue of scoring reliability is critically important to any future use of CMA.and third-year residents.02 Concept link 178 (72) 223 (83) . Despite the small sample size. as reflected in the significantly higher cross-link scores of second. We found that interrater reliability was moderate to strong and.01 Hierarchy 65 (20) 61 (22) . This change is perhaps best illustrated by qualitative comparison of the preinstruction and postinstruction maps (eg. Thus. No.44 Examples 65 (32) 57 (23) . 2000—Vol 284. September 6. Comparison of Total Concept Map and Subscores Before and After Educational Intervention* PrePostinstruction instruction P Component Map Map Value† Total 429 (119) 516 (196) . In reviewing this litera- Table 2.and third-year residents were significantly higher than those of firstyear residents. Quantitatively.23 Therefore. The vast majority of previous research regarding concept mapping has involved high school and college science education. the absence of a positive correla- Figure 4. these scores did not correlate well with CMA scores. It is important to note that this difference narrowed. residents who scored highly on CMA were not the same residents who scored highly on the ITE examination. further differentiation of existing concepts. See “Scoring Method” subsection for explanation of scoring method.

29. Davis.35:1015-1029. ed. CMA has consistently been found to measure conceptual change in situations in which such change would be expected and to detect differences between students with more training in a particular field compared with those with less. Novak JD. Albertazzi PG. 2. Concept mapping to facilitate veterinary students’ understanding of fluid and electrolyte disorders. J Chem Ed. Acad Med. Concept mapping: a useful tool for science education. 1993.67: 625-645. Dissertation Abstracts Int. Acknowledgment: We thank Luann Cruys. 5. 30. Edmondson KM. Sci Educ. Royer J.40:631-666. Markham KM. Strike KA.22. 1978.10: 21-33. Novak JD. MD. Ausubel DP. Use of concept mapping to define learning objectives in a master of public health program. Future research must focus on further validating this method and addressing issues of reliability before CMA can be applied on a larger scale. Concept mapping brings long-term movement toward meaningful learning. University of California. 26.71(suppl):S13-S15. A twelve year study of science concept learning. Learning pulmonary physiology: comparison of student and faculty knowledge structures.7:73-81. 12. 1989. Jones MG. other investigators have used the same strategy to build a case for the validity of CMA. Gowin DB. Using explanatory knowledge during collaborative problem solving in science. 6. 20. 2000. San Diego. Individualized Science: Like It Is. student Pathfinder networks became more similar to the networks generated by faculty experts. J Res Sci Teach. 1985. probably because there is no criterion standard measure of conceptual framework to which CMA can be compared. We have provided encouraging preliminary evidence supporting the validity of CMA as a way to measure conceptual change and differences among resident physicians.73:10841088. 46:2622A. 1990. 1998. New pathways in general medical education. The Language and Thought of the Child. 1990. 32. Philipps. 1981. 1996. Am Biol Teach. Funding/Support: This work was supported in part by grants from the Children’s Miracle Network Research Fund and the Hibbard E. 1998. Novak JD. 1999. 9 (Reprinted) ©2000 American Medical Association.48:2590A. and logistical practicality. N Engl J Med. 1988. 1996. 8. Annu Rev Psychol. Problems and issues in the use of concept maps in science assessment. Sci Educ. NY: Grune & Stratton. Coleman EB. 1991. 1994.36:475-492. Mass. J Res Sci Teach. Nevertheless.16-18. Acad Med. J Res Sci Teach. 27.66:211-227. provide additional evidence suggesting the predictive validity of CMA. Hanesian H. Davis. 1992.28: 117-153. Chastonay P. Duit R. Debbie Barnes. Bar-Lavie BZ. September 6. Sonak B. 3. Applying learning psychology and philosophy of science to biology teaching. dents organize and use knowledge in a way that traditional objective tests cannot. et al. Bodner GM. We also are indebted to Constance Bowe. Concept map assessment of classroom learning: reliability. Techniques and procedures for assessing cognitive skills. Cisero CA. Carlo MS. Objective Structured Clinical Examinations). Smith DF. 11:21-25.63:873-878. Lindbloom-Ylaenne S. Mintzes JJ. J Res Sci Teach. J Res Sci Teach. 3rd ed. Musonda D. 1982. 9. Novak JD. Tosteson DC. Cambridge. 7. for critical review of the manuscript. Spontaneous study strategies and the quality of knowledge construction. Edmondson KM. Novak JD. Lonka K. 10. New York. 21. Gertzog WA. Finally. 1984:15-54. Lonning RA. concept mapping. Novak JD. 1998. Teaching Science for Understanding: A Constructivist View. Piaget J. No. Mahwah. Novak JD. McClure JR. Ruiz-Primo MA. DC: American Association for the Advancement of Science Press. Learning Science in the Schools: Research Reforming Practice. Markow PG.CONCEPT MAPPING ASSESSMENT professionals. 2nd ed. Hewson PW. 1972:14-30. 14. Usefulness of concept maps in college chemistry laboratories: students’ perceptions and effects on achievement. and while not conclusive. after instruction. Williams Research Fund at the University of California. and educating. Should concept maps be scored numerically. 31. 1983. 15. Shavelson RJ. J College Sci Teach. Novak JD. These findings appear consistent with our study. It also could provide a tool for educators to identify unique distortions in students’ understanding of content and to identify errors of omission.43:13-20. MD. 1999. Previous Presentation: Presented in part at the Pediatric Academic Societies Joint Meeting.27:937-949. Glaser R. 1963. Novak JD. Constructivism: a theory of knowledge. Shafer TH. 11. Suen HK. McGaghie WC. validity. Our findings also indicate that rater training is critical to the use of CMA. CMA has the potential to evaluate how students or resiTriezenberg H. 16. Wandersee JH.74:461-472.69:108-110. Stuart HA.31. 1997. Papart JP. 23. evidence supporting the validity of concept mapping as an assessment tool is much more limited. Br J Educ Psychol. Washington. Concept maps and the development of cases for problem-based learning. Concept maps in chemistry education. CMA might provide insight into why some residents score well on objective written examinations but have difficulty applying this knowledge to clinical situations. 1110 JAMA. Dunn CS. von Glaserfeld E. NJ: Lawrence Erlbaum Associates. Laporte JD. 1996. Mintzes JJ. 33. 1998. Mintzes JJ.com at Penn State Milton S Hershey Med Ctr on November 10. The Psychology of Meaningful Verbal Learning. May 16. 2000—Vol 284. For example. J Learn Sci. Washington. the investigators found that. Department of Pediatrics. Boston. Concept mapping in college science teaching. Dissertation Abstracts Int. J Chem Educ. The use of concept mapping and knowledge vee mapping with junior high school science students. 1995: 229-245. Learning How to Learn. Sci Educ. Boerger RL. Posner GJ. Johansen G. 28.32 In medical education. In: Glynn SM. 1959. New York. 13. Eur J Sci Educ. All rights reserved. Roletto E. in a variety of secondary school and undergraduate settings. England: Routledge & Kegan Paul. Teach Learn Med. The concept map as a research and evaluation tool: further evidence of validity. 1994. Calif: Academic Press. we thank the pediatric residents at UC Davis for participating in this study and their willingness to explore a new way of thinking about assessment. and Sally Colvin for invaluable administrative support without which this study could not have been completed and Anthony F. was used to assess student understanding of pulmonary physiology.7:387-427. a substantially different form of concept mapping.14:117-121. for her dedication to medical education and developing and teaching the seizure education course. Traditional measures of competence. such as board scores or even newer competency-based assessments (eg. The Practice of Constructivism in Science and Mathematics Education.24-30 However.31:91-101. Novak JD. Educational Psychology: A Cognitive View. Audio-tutorial techiques for individualized science instruction in the elementary school. 19. interviewing. Arnaudin MW. 1986. 17.63:201-224. Regis A. Bassok M.322:234-238. NY: Holt Rinehart & Winston. Learning theory and the study of instruction. 2008 . 1990. may not represent appropriate comparisons since they shed little light on how a learner thinks about a problem. 24.67: 13-24. Enhancing meaningful learning in an environmental education program: a case study of class empowered through the use of Novak’s and Gowin’s principles of learning how to learn. Concept mapping as an approach to assessment of students’ representation of structural knowledge. Downloaded from www. London. 1984. Ausubel DP.33 In that study. 18. 4. Teach Learn Med. Heinze-Fry JA. Department of Neurology.16-18. we identified only a REFERENCES 1. England: Cambridge University Press. Rev Educ Res. eds. Concept mapping: a strategy for organizing knowledge. Beyerbach BA.12 Thus. Accommodation of a scientific conception: toward a theory of conceptual change. Ravitch MM. Finally.jama. 25. Gowin DB. 1986. DC: National Science Teachers Association. In: single related study in which Pathfinder networks. 22. McCrimmon DR.14. Lahtinen V. Am Educ Res J.33:569-600.