George Lakoff The Neural Theory of Language Project (How Can the Brain Compute Concepts?) [Jan.

16 1998 - CS200] How do human neural systems learn the specific kinds of concepts that occur in natural languages and the language that expresses those concepts? This is the question taken up in the Neural Theory of Language (NTL) research group at the International Computer Science Institute at Berkeley, a collaboration since the late 1980's between Jerome Feldman, George Lakoff, and their students. At the heart of the modeling effort is Feldman's idea of structured connectionism, which can be used to model highly specific brain structures. The central enterprise of the group is to provide neural models of embodied cognition, especially the acquisition and use of language and thought as described in cognitive linguistics. This talk will be an overview of three modeling efforts done in dissertations by students in the group: Terry Regier's thesis on the acquisition of spatial relations concepts and terms. David Bailey's thesis on the learning of verbs of hand motion. And Srini Narayanan's thesis on how aspect arises in neural control systems, on how metaphor is modeled, and on how the same neural control system that can control body movements can do abstract reasoning. Since there is an enormous gap between the physical brain and the level of human concepts and language, the NTL group has developed a paradigm for ultimately bridging that gap in a small set of precise steps, using research methodologies already in place within cognitive science. Level 1. Cognitive Science and Cognitive Linguistics Level 2. Neurally Reducible Conventional Computational Models Level 3. Structured Connectionist Models Level 4. Computational Neuroscience Level 5. Neuroscience The link between Level 4 and Level 5 is given by computational neuroscience, which models the brain as if it were "circuitry," with axons and dendrites seen as "connections," activation and inhibition as positive and negative numerical values, and so on. Since brain circuitry is enormously complicated, structured connectionist models seek a simplified representation of such circuitry, in which equivalent computations are carried out by neural circuitry of minimal complexity. Thus, the link between Level 3 and Level 4 is one of simplification: the minimal structured circuitry that will do the same job as models of the actual "brain circuitry." Sometimes the relation between the level of analysis in cognitive science and cognitive linguistics and the structured connectionist level can be given directly, as in the model by Regier. But even the simplest neural models that carry out a complex task can be so complicated that an intermediate level of representation is helpful in doing the modeling. In the NTL paradigm, conventional computational systems from computer science are sought out that have the basic properties of neural systems: parallel operation, distributed control (no internal clock or centralized controller), ability to react quickly and effectively to changing contexts, resource dependency, ability to learn via statistical correlation, and so. Such

conventional computational mechanisms can form a link between level 2 and level 3. They are only used when it is known how to map them directly to structured connectionist neural models.Members of the group have constructued novel forms of conventional-style computational along these guidelines. Only neurally reducible computational models are used. Very few conventional computational systems satisfy all the requirements for neural reduction. Such models help us to get a precise handle on how conceptual structure is embodied, esopecially for the very specific, detailed conceptual structures that cognitive linguists have uncovered. more information on this research: The Neural Theory of Language Project Srini Narayanan A Dynamic Model of Metaphoric Reasoning about Events [Jan. 23 1998 - CS200] This talk describes a new computational model for metaphoric reasoning about event structure. A novel feature of the model is an "active" representation of embodied motion and manipulation verbs (such as walk, push, slide, slip) that is inspired by what is known about high-level sensory-motor control and satisfies general constraints on modeling neural activity. As a result, we are able to use a single representation for controlling actions in a dynamic environment and for context-sensitive, simulative inference in language understanding. Monitoring and control parameters abstracted from the basic model provide semantic grounding for interpreting aspectual expressions and seem to offer elegant ways to solve the vexing linguistic problem of aspectual composition. The implemented model is able to use metaphoric projections of motion verbs to infer in real-time important features of abstract plans and events; potentially explaining the frequent use of motion and manipulation terms in discourse about abstract plans. Results of applying our model on discourse fragments from newspaper stories in international economics show that crucial facts about abstract plans, goals, resources and intent are communicated by projections from embodied concepts. DIRECTLY RELEVANT PAPERS -----------------------Narayanan, Srini (1996). Embodiment in Language Understanding: Modeling the semantics of causal narratives''. AAAI Symposium on Embodied Cognition and Action, Nov. 9-11, Cambridge Mass. Technical Report, FS-96-02, AAAI Press, Menlo Park, CA. Narayanan, Srini (1997). Talking The Talk Is Like Walking the Walk: A Computational Model of Verbal Aspect''. Proceedings of the Nineteenth Annual Meeting of the Cognitive Science Society Aug 9-11, Stanford: Stanford University Press, 1997. Narayanan, Srini (1997). KARMA: Knowledge-Based Active Representations For Metaphor and Aspect''. Ph.D. Dissertation,

Computer Science Division, University of California, Berkeley, 1997. NTL PROJECT OVERVIEW PAPERS --------------------------Feldman, J, G. Lakoff, D. Bailey, S. Narayanan, T. Regier, A. Stolcke (1996). Lzero: The First Five Years''. Artificial Intelligence Review, v10 103-129, April 1996. Bailey, D., J. Feldman, S. Narayanan, and G. Lakoff (1997). Embodied Lexical Development'', Proceedings of the Nineteenth Annual Meeting of the Cognitive Science Society, Aug 9-11, Stanford: Stanford University Press, 1997. postscript versions of readings (ICSI) Gilles Fauconnier Conceptual Integration Networks [Jan. 30 1998 - CS200] Conceptual integration is on a par with other high level cognitive processes such as analogy, induction, recursion, reference-point organization, conceptual metaphor, and it interacts with all of them. It has systematic dynamic mapping properties, obeys optimality constraints, and operates, with the same structural characteristics, in a wide variety of cognitive phenomena, such as synchronic and diachronic meaning formation, problem solving, action and design, scientific evolution, humor, counterfactual reasoning. The study of this cognitive operation sheds light on creative processes of conceptual change with emergent structure in short term everyday human interaction, as well as in long term evolution of thought. Conceptual integration networks are constructed non- compositionally in accordance with optimality principles that may compete with each other. Six guiding principles have been proposed (integration, web, unpacking, topology, good reason, and metonymy tightening). Emergent properties of conceptual blends. which account for many types of creativity, are constrained by such principles. When viewed in terms of conceptual integration networks, superficially quite different types of meaning construction (e.g. Fregean predicate structure, counterfactuals, frame blends, metaphors) all fall on a single continuum. Moreover, although the meanings are not truth-conditionally compositional, the mapping schemes are compositional. This reveals an underlying unity of meaning construction processes that cuts across the traditional distinctions between core logical semantics, pragmatics, figurative language, and rhetoric. The research ties in with work in cognitive science on analogy, metaphor, framing, and grammar, and extremely interesting applications have been suggested in music, literature, sign language, political science, and mathematics. Integration is commonly presented as a purely mental phenomenon, but as Ed Hutchins has shown, we often make use of structure available in the world, or culturally created, in order to anchor blends. Novel technology and design typically involve integration, as in computer interfaces. More subtle is the kind of grounding found in language use, and especially in sign language. Scott Liddell has shown how meaning in ASL gets constructed through on-line

blending. Here, integration with material anchors is part of everyday language use, with anchors provided either by the real environment (including the speech participants), or by the construction of a surrogate space (with invisible surrogates added to the real environment). Uniform principles of integration operate both at high levels of scientific, artistic, and literary thought and at lower levels of everyday thinking, talking, and understanding. This calls into question the sharp distinctions of nature often made between ordinary syntax and meaning, problem-solving and reasoning, scientific discovery, artistic and literary creation. References Coulson, Seana. 1997. Semantic Leaps: Frame-Shifting and Conceptual Blending. UCSD Ph. D. dissertation. Coulson, S. and G. Fauconnier. (submitted). Fake Guns and Stone Lions: Conceptual Blending and Privative Adjectives. [for Conceptual Structure, Discourse, and Language, III] Fauconnier, G. & Turner, M. (in press). Conceptual Integration Networks. Cognitive Science. Liddell, Scott K. 1995. Real, Surrogate and Token Space: grammatical consequences in ASL. In K. Emmorey and J. Reilly (eds.), Language, Gesture,and Space. Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers, 19-41. Liddell, Scott. 1997. Grounded Blends, Gesture and Meaning in American Sign Language Discourse. Fifth International Cognitive Linguistics Conference. 1997. Mandelblit, Nili. 1997. Creativity and Schematicity in Grammar and Translation: The Cognitive Mechanisms of Blending. Doctoral dissertation, UCSD Cognitive Science. Robert, A. (in press) Blending in Mathematical Proofs. Conceptual Discourse, Structure, and Language II. Ed. Jean-Pierre Koenig. Stanford: Center for the Study of Language and Information Sweetser, Eve. 1997. "Mental Spaces and Cognitive Linguistics: A Cognitively Realistic Approach to Compositionality." Fifth International Cognitive Linguistics Conference. 1997. Turner, Mark. 1996a. Conceptual Blending and Counterfactual Argument in the Social and Behavioral Sciences in Philip Tetlock and Aaron Belkin, editors, Counterfactual Thought Experiments in World Politics (Princeton: Princeton University Press). Zbikowski, Lawrence. 1997. "Conceptual blending and song." Manuscript. University of Chicago. more on blending and conceptual integration Ed Hutchins Material anchors for conceptual blends [Feb. 6 1998 - CS200]

An interesting class of cognitive processes utilize mappings between conceptual spaces and physical spaces. This talk considers a number of examples in which physical structures act as material anchors for complex assemblages of conceptual relations. In these processes, the spatial organization of material anchors plays a variety of roles including support for conceptual blending, the substitution of robust perceptual processes for conceptual processes, and overcoming the limits of short-term memory. The examples range from everyday devices such an analog wristwatch to specialized tools such as aircraft instruments and the complex imagery used by micronesian navigators. -------The two readings are a paper I wrote for the journal Cognitive Science titled "How a cockpit remembers its speeds" and the second chapter of my recent book "Cognition in the Wild." I intend the cockpit speeds paper to be background for a discussion of the airspeed indicator as a material anchor for a blend of space and speed to yield a blended speed-space in which perceptual operations on spatial relations of elements of the display stand in for conceptual operations concerning speed. This is an emergent property of the blend realized in material form. The chapter from Cognition in the Wild contains two examples that I will discuss in the talk. One is the system of micronesian navigation in which navigators in the western Pacific ocean use complex imagery that superimposes several conceptual frames into a single blended space. The other is the way Medieval navigators in northern Europe figured out how to look at a compass rose and see the tides there.

Keith Holyoak Dept. of Psychology University of California, Los Angeles Relational Reasoning in a Physical Symbol System [Feb. 13 1998 - CS200] Is the human mind a physical symbol system, and if so what does this imply for theories of reasoning and learning? I will argue that the mind is symbolic in a way that requires dynamic variable binding over distributed representations of meaning. Neither PDP-style connectionist models nor traditional symbolic models satisfy these twin requirements. These requirements are met by symbolic connectionism, an approach I will illustrate using the LISA model of analogical reasoning. Readings John Hummel and Keith Holyoak. 1997. Distributed Representations of Structure: A Theory of Analogical Access and Mapping. Psychological Review

Vol. 104, no. 3, 427-466. Keith Holyoak and John Hummel. in press. The Proper Treatment of Symbols in a Connectionist Architecture. In E. Deitrich & A. Markman (Eds.), Cognitive dynamics: Conceptual change in humans and machines. Cambridge, MA: MIT Press. John Hummel and Keith Holyoak. 1996. LISA: A Computational Model of Analogical Inference and Schema Induction. 18th CogSci Society (352-357). Review of 'Mental Leaps' (by Doug Hofstadter). AI Magazine. 1995.

Seana Coulson Dept. of Psychology University of Arizona To be real: Representation and misrepresentation in models of concept combination [Feb. 20 1998 - CS200] I review a number of models of the concept combination coded by simple noun phrases such as "brown cow," "chocolate bunny," and "fake ID". While many models seem to be able to explain how people combine concepts to yield "brown cow," combinations such as "chocolate bunny" and "fake ID" have proven more challenging. I suggest that these limitations result from three things: first, the assumption that concepts are static structures in long term memory; second, commitment to the idea of a conceptual core; and third, the idea that concept combination is the algorithmic combination of objective features. In contrast, I suggest that concepts are dynamically assembled in response to contextual needs, and that relational mapping operations are fundamental to concept combination. I present a variety of data which support this position and show how a comprehensive theory of concept combination can be developed from conceptual integration theory (Fauconnier & Turner, in press). Overall, these findings suggest that the mapping operations underlying concept combination are dependent on (i) causal information, (ii) relational information, and, even, (iii) human theory of mind. Readings: Coulson, S. 1996. Menendez Brothers Virus: Blended Spaces and Internet Humor. In A. Goldberg, ed., Conceptual Structure, Discourse, and Language. CSLI Publications. Coulson, S. and G. Fauconnier. ms. Fake Guns and Stone Lions: Conceptual Blending and Privative Adjectives. Franks, B. 1995. Sense Generation: A "quasi-classical" approach to concepts and concept combination. Cognitive Science 19:441-505.

Arthur Markman (Columbia) Cognitive models of analogy, similarity, and metaphor [Feb. 27 1998 - CS200] There has been a remarkable degree of convergence among researchers in the last ten years about how to think about the process of analogical mapping in humans. In particular, the field has generally agreed that analogical comparisons involve finding common relational structures across domains. Based on the shared grounding assumptions, a number of computational models of analogy have been developed including Falkenhainer, Forbus, & Gentner's SME, Hummel & Holyoak's LISA, Keane's IAM, Holyoak & Thagard's ACME and Hofstadter and Mitchell's COPYCAT. These models differ from each other in the particular constraints on analogical mapping they value and on the architectures they use to create analogical matches. The shared foundational assumptions among researchers have allowed the research community to define important disagreements about aspects of the mapping process. We will begin with a discussion of the points of agreement among researchers. Then, we will focus on key areas of disagreement including: (1) the importance of structure in analogical mapping, (2) the role of semantic similarity in mapping, (3) the development of representations, and (4) the nature of representational change. In order to examine these disagreements we will explore the computational models and also discuss empirical evidence from studies of analogy, similarity and metaphor processing in humans. Suggested Readings Forbus, K. D., Gentner, D., & Law, K. (1995). MAC/FAC: A model of similaritybased retrieval. Cognitive Science, 19(2), 141-205. Forbus, K. D., Gentner, D., Markman, A. B., & Ferguson, R. W. (in press). Analogy just looks like high level perception: Why a domain-general approach to analogical mapping is right. Journal of Experimental and Theoretical Artificial Intelligence. Gentner, D., & Markman, A. B. (1997). Structural alignment in analogy and similarity. American Psychologist, 52(1), 45-56. Gentner, D. & Wolff, P. (in press). Metaphor and knowledge change. To appear in E. Dietrich, & A. B. Markman (Eds.) Cognitive dynamics. Cambridge, MA: The MIT Press. Markman, A. B. (in press). Constraints on analogical inference. Cognitive Science. Additional (background) readings Gentner, D., & Wolff, P. (1997). Alignment in the processing of metaphor. Journal of Memory and Language, 37, 331-355. [Empirical evidence about metaphor processing.]

Hummel, J. E., & Holyoak, K. J. (1997). Distributed representations of structure: A theory of analogical access and mapping. Psychological Review, 104(3), 427-466. [An introduction to LISA] Keane, M. T. Ledgeway, T., & Duff, S. (1994). Constraints on analogical mapping: A comparison of three models. Cognitive Science, 18, 387-438. [An introduction to IAM, and a comparison of IAM, SME and ACME.]

Ray Gibbs Dept. of Psychology University of California Santa Cruz Embodied Metaphor In Language, Thought, And Culture [March 6 1998 - CS200] My talk will focus on how ordinary and poetic language reveals significant insights into the embodied character of metaphor in language, thought, and culture. I will describe how systematic analyses of different linguistic patterns (e.g., in word meanings, conventional expressions, and novel, poetic discourse) suggest the importance of embodiment in how people make sense of different aspects of their experience, and provide partial motivation for the ways people speak about these experiences. Special attention will be given to looking at how recurring patterns of embodied experience give rise to "image schemas" that can be metaphorically elaborated upon to help structure many basic abstract concepts. I will describe different aspects of my research program looking at how embodied metaphor influences people's interpretations and on-line comprehension of word meanings, conventional expressions, and poetry. Following this, I suggest several ways that this empirical research constrains theories of meaning and concepts, particularly in regard to acknowledging the necessity of metaphor and embodiment in both language and thought. Finally, I will argue that cognitive scientists must be very careful NOT to assume that just because metaphor is indeed conceptual (and not only a linguistic device), it must necessarily be viewed as another kind of internal mental representation. An important part of my present research strategy is to explore the ways that metaphor is "off-loaded" into the cultural world. Suggested Readings

*Gibbs, R. (1996). Why many concepts are metaphorical. "Cognition" 61, 309-319. Gibbs, R. (1994). "The poetics of mind: Figurative thought, language, and understanding." New York: Cambridge University Press. (see chapters 1, *4, and 6). Gibbs, R., Beitel, D., Harrington, M., & Sanders, P. (1995). Taking a stand on the meanings of "stand": Bodily experience as motivation for polysemy. "Journal of Semantics, " 11, 231-251. *Gibbs, R. (1997). How language reveals the embodied nature of creative cognition. In T. Ward, S. Smith, and J. Vaid (Eds.), "Creative thought." Washington, DC: American Psychological Association. Gibbs, R., Bogdonovich, J., Sykes, J., & Barr, D. (1997). Metaphor in idiom comprehension. "Journal of Memory and Language," 37, 141-154. Douglas Hofstadter Center for Research on Concepts and Cognition Indiana University, Bloomington At the Core of Human Creativity... [March 13 1998 - CS200] Using examples from the worlds of art and literature, I will attempt to describe how a polished creation emerges out of a broth of many diverse cognitive acts: internalization, essence-seeking, quality judgment, flexing, exporting, adapting, uniformity-imposition, symmetry-favoring, echoinsertion, tracks-covering, and others. These kinds of mechanisms -- as well as a manner of allowing them all to coexist and to exert their influences in parallel on developing creations -have been implemented, albeit in a rudimentary manner, in computer programs developed by my associates and myself over the past couple of decades. All of them operate in micro-worlds that anyone can understand and relate to, and in which, by great fortune, the key cognitive issues also emerge with maximal clarity. The work is ongoing, and one important avenue of progress is the insertion of greater self-awareness into such systems, so that they are more "conscious" of what they are doing, and thus what they do is more genuinely "under their control". The focus of our investigations is the nature of esthetic creation, with all the vagueness and subjectivity that that inevitably entails. Our goal might be characterized as that of describing in somewhat objective terms the underpinnings of the seemingly intangible and murky yet very real and powerful subjective impulses that always lie behind any serious creative effort.

========= Readings: Chapters 4, 5, 6, 7 of Hofstadter, Douglas R. and the Fluid Analogies Research Group. 1995. Fluid Concepts and Creative Analogies: Computer Models of the Fundamental Mechanisms of Thought. Basic Books.