Natural-language understanding

Natural-language understanding (NLU) or natural-language interpretation (NLI)[1] is a subtopic of

natural-language processing in artificial intelligence that deals with machine reading comprehension.
Natural-language understanding is considered an AI-hard problem.[2]

There is considerable commercial interest in the field because of its application to automated reasoning,[3]
machine translation,[4] question answering,[5] news-gathering, text categorization, voice-activation,
archiving, and large-scale content analysis.

History
The program STUDENT, written in 1964 by Daniel Bobrow for his PhD dissertation at MIT, is one of the
earliest known attempts at natural-language understanding by a computer.[6][7][8][9][10] Eight years after
John McCarthy coined the term artificial intelligence, Bobrow's dissertation (titled Natural Language Input
for a Computer Problem Solving System) showed how a computer could understand simple natural
language input to solve algebra word problems.

A year later, in 1965, Joseph Weizenbaum at MIT wrote ELIZA, an interactive program that carried on a
dialogue in English on any topic, the most popular being psychotherapy. ELIZA worked by simple parsing
and substitution of key words into canned phrases and Weizenbaum sidestepped the problem of giving the
program a database of real-world knowledge or a rich lexicon. Yet ELIZA gained surprising popularity as a
toy project and can be seen as a very early precursor to current commercial systems such as those used by
Ask.com.[11]

In 1969, Roger Schank at Stanford University introduced the conceptual dependency theory for natural-
language understanding.[12] This model, partially influenced by the work of Sydney Lamb, was
extensively used by Schank's students at Yale University, such as Robert Wilensky, Wendy Lehnert, and
Janet Kolodner.

In 1970, William A. Woods introduced the augmented transition network (ATN) to represent natural
language input.[13] Instead of phrase structure rules ATNs used an equivalent set of finite state automata
that were called recursively. ATNs and their more general format called "generalized ATNs" continued to
be used for a number of years.

In 1971, Terry Winograd finished writing SHRDLU for his PhD thesis at MIT. SHRDLU could
understand simple English sentences in a restricted world of children's blocks to direct a robotic arm to
move items. The successful demonstration of SHRDLU provided significant momentum for continued
research in the field.[14][15] Winograd continued to be a major influence in the field with the publication of
his book Language as a Cognitive Process.[16] At Stanford, Winograd would later advise Larry Page, who
co-founded Google.

In the 1970s and 1980s, the natural language processing group at SRI International continued research and
development in the field. A number of commercial efforts based on the research were undertaken, e.g., in
1982 Gary Hendrix formed Symantec Corporation originally as a company for developing a natural
language interface for database queries on personal computers. However, with the advent of mouse-driven
graphical user interfaces, Symantec changed direction. A number of other commercial efforts were started
around the same time, e.g., Larry R. Harris at the Artificial Intelligence Corporation and Roger Schank and
his students at Cognitive Systems Corp.[17][18] In 1983, Michael Dyer developed the BORIS system at
Yale which bore similarities to the work of Roger Schank and W. G. Lehnert.[19]

The third millennium saw the introduction of systems using machine learning for text classification, such as
the IBM Watson. However, experts debate how much "understanding" such systems demonstrate: e.g.,
according to John Searle, Watson did not even understand the questions.[20]

John Ball, cognitive scientist and inventor of Patom Theory (http://www.isi.hhu.de/fileadmin/redaktion/Oef

fentliche_Medien/Fakultaeten/Philosophische_Fakultaet/Sprache_und_Information/Ball_PatomTheory.pd
f), supports this assessment. Natural language processing has made inroads for applications to support
human productivity in service and ecommerce, but this has largely been made possible by narrowing the
scope of the application. There are thousands of ways to request something in a human language that still
defies conventional natural language processing. "To have a meaningful conversation with machines is
only possible when we match every word to the correct meaning based on the meanings of the other words
in the sentence – just like a 3-year-old does without guesswork."

Scope and context

The umbrella term "natural-language understanding" can be applied to a diverse set of computer
applications, ranging from small, relatively simple tasks such as short commands issued to robots, to highly
complex endeavors such as the full comprehension of newspaper articles or poetry passages. Many real-
world applications fall between the two extremes, for instance text classification for the automatic analysis
of emails and their routing to a suitable department in a corporation does not require an in-depth
understanding of the text,[21] but needs to deal with a much larger vocabulary and more diverse syntax than
the management of simple queries to database tables with fixed schemata.

Throughout the years various attempts at processing natural language or English-like sentences presented to
computers have taken place at varying degrees of complexity. Some attempts have not resulted in systems
with deep understanding, but have helped overall system usability. For example, Wayne Ratliff originally
developed the Vulcan program with an English-like syntax to mimic the English speaking computer in Star
Trek. Vulcan later became the dBase system whose easy-to-use syntax effectively launched the personal
computer database industry.[22][23] Systems with an easy to use or English like syntax are, however, quite
distinct from systems that use a rich lexicon and include an internal representation (often as first order logic)
of the semantics of natural language sentences.

Hence the breadth and depth of "understanding" aimed at by a system determine both the complexity of the
system (and the implied challenges) and the types of applications it can deal with. The "breadth" of a
system is measured by the sizes of its vocabulary and grammar. The "depth" is measured by the degree to
which its understanding approximates that of a fluent native speaker. At the narrowest and shallowest,
English-like command interpreters require minimal complexity, but have a small range of applications.
Narrow but deep systems explore and model mechanisms of understanding,[24] but they still have limited
application. Systems that attempt to understand the contents of a document such as a news release beyond
simple keyword matching and to judge its suitability for a user are broader and require significant
complexity,[25] but they are still somewhat shallow. Systems that are both very broad and very deep are
beyond the current state of the art.

Components and architecture

Regardless of the approach used, most natural-language-understanding systems share some common
components. The system needs a lexicon of the language and a parser and grammar rules to break
sentences into an internal representation. The construction of a rich lexicon with a suitable ontology
requires significant effort, e.g., the Wordnet lexicon required many person-years of effort.[26]

The system also needs theory from semantics to guide the comprehension. The interpretation capabilities of
a language-understanding system depend on the semantic theory it uses. Competing semantic theories of
language have specific trade-offs in their suitability as the basis of computer-automated semantic
interpretation.[27] These range from naive semantics or stochastic semantic analysis to the use of
pragmatics to derive meaning from context.[28][29][30] Semantic parsers convert natural-language texts into
formal meaning representations.[31]

Advanced applications of natural-language understanding also attempt to incorporate logical inference

within their framework. This is generally achieved by mapping the derived meaning into a set of assertions
in predicate logic, then using logical deduction to arrive at conclusions. Therefore, systems based on
functional languages such as Lisp need to include a subsystem to represent logical assertions, while logic-
oriented systems such as those using the language Prolog generally rely on an extension of the built-in
logical representation framework.[32][33]

The management of context in natural-language understanding can present special challenges. A large
variety of examples and counter examples have resulted in multiple approaches to the formal modeling of
context, each with specific strengths and weaknesses.[34][35]

See also
Computational semantics
Computational linguistics
Discourse representation theory
Deep linguistic processing
History of natural language processing
Information extraction
Mathematica[36][37][38]
Natural-language processing
Natural-language programming
Natural-language user interface
Siri (software)
Wolfram Alpha
Open information extraction
Part-of-speech tagging
Speech recognition

Notes
1. Semaan, P. (2012). Natural Language Generation: An Overview (http://www.lacsc.org/paper
s/PaperA6.pdf). Journal of Computer Science & Research (JCSCR)-ISSN, 50-57
 2. Roman V. Yampolskiy. Turing Test as a Defining Feature of AI-Completeness . In Artificial
Intelligence, Evolutionary Computation and Metaheuristics (AIECM) --In the footsteps of Alan
Turing. Xin-She Yang (Ed.). pp. 3-17. (Chapter 1). Springer, London. 2013.
http://cecs.louisville.edu/ry/TuringTestasaDefiningFeature04270003.pdf
3. Van Harmelen, Frank, Vladimir Lifschitz, and Bruce Porter, eds. Handbook of knowledge
representation (https://books.google.com/books?id=xwBDylHhJhYC&q=%22natural+langua
ge+understanding%22). Vol. 1. Elsevier, 2008.
4. Macherey, Klaus, Franz Josef Och, and Hermann Ney. "Natural language understanding
using statistical machine translation (https://www.researchgate.net/profile/Klaus_Macherey/p
ublication/2371092_Natural_Language_Understanding_Using_Statistical_Machine_Transl
ation/links/00463523b62bc9b5e6000000/Natural-Language-Understanding-Using-Statistica
l-Machine-Translation.pdf)." Seventh European Conference on Speech Communication and
Technology. 2001.
5. Hirschman, Lynette, and Robert Gaizauskas. "Natural language question answering: the
view from here (https://www.researchgate.net/profile/Rob_Gaizauskas/publication/23180719
5_Natural_Language_Question_Answering_The_View_from_Here/links/0c96052a09fa7b8
19e000000/Natural-Language-Question-Answering-The-View-from-Here.pdf)." natural
language engineering 7.4 (2001): 275-300.
6. American Association for Artificial Intelligence Brief History of AI [1] (http://www.aaai.org/AITo
pics/pmwiki/pmwiki.php/AITopics/BriefHistory)
7. Daniel Bobrow's PhD Thesis Natural Language Input for a Computer Problem Solving
System (http://hdl.handle.net/1721.1/5922).
8. Machines who think by Pamela McCorduck 2004 ISBN 1-56881-205-1 page 286
9. Russell, Stuart J.; Norvig, Peter (2003), Artificial Intelligence: A Modern Approach Prentice
Hall, ISBN 0-13-790395-2, http://aima.cs.berkeley.edu/, p. 19
10. Computer Science Logo Style: Beyond programming by Brian Harvey 1997 ISBN 0-262-
58150-7 page 278
11. Weizenbaum, Joseph (1976). Computer power and human reason: from judgment to
calculation W. H. Freeman and Company. ISBN 0-7167-0463-3 pages 188-189
12. Roger Schank, 1969, A conceptual dependency parser for natural language Proceedings of
the 1969 conference on Computational linguistics, Sång-Säby, Sweden, pages 1-3
13. Woods, William A (1970). "Transition Network Grammars for Natural Language Analysis".
Communications of the ACM 13 (10): 591–606 [2] (http://www.eric.ed.gov/ERICWebPortal/cu
stom/portlets/recordDetails/detailmini.jsp?_nfpb=true&_&ERICExtSearch_SearchValue_0=
ED037733&ERICExtSearch_SearchType_0=no&accno=ED037733)
14. Artificial intelligence: critical concepts, Volume 1 by Ronald Chrisley, Sander Begeer 2000
ISBN 0-415-19332-X page 89
15. Terry Winograd's SHRDLU page at Stanford SHRDLU (http://hci.stanford.edu/~winograd/shr
dlu/)
16. Winograd, Terry (1983), Language as a Cognitive Process, Addison–Wesley, Reading, MA.
17. Larry R. Harris, Research at the Artificial Intelligence corp. ACM SIGART Bulletin, issue 79,
January 1982 [3] (http://portal.acm.org/citation.cfm?id=1056663.1056670)
18. Inside case-based reasoning by Christopher K. Riesbeck, Roger C. Schank 1989 ISBN 0-
89859-767-6 page xiii
19. In Depth Understanding: A Model of Integrated Process for Narrative Comprehension..
Michael G. Dyer. MIT Press. ISBN 0-262-04073-5
20. Searle, John (23 February 2011). "Watson Doesn't Know It Won on 'Jeopardy!' " (https://ww
w.wsj.com/articles/SB10001424052748703407304576154313126987674). Wall Street
Journal.
21. An approach to hierarchical email categorization by Peifeng Li et al. in Natural language
processing and information systems edited by Zoubida Kedad, Nadira Lammari 2007
ISBN 3-540-73350-7
22. InfoWorld, Nov 13, 1989, page 144
23. InfoWorld, April 19, 1984, page 71
24. Building Working Models of Full Natural-Language Understanding in Limited Pragmatic
Domains by James Mason 2010 [4] (http://www.yorku.ca/jmason/UnderstandingEnglishInLi
mitedPragmaticDomains.html)
25. Mining the Web: discovering knowledge from hypertext data by Soumen Chakrabarti 2002
ISBN 1-55860-754-4 page 289
26. G. A. Miller, R. Beckwith, C. D. Fellbaum, D. Gross, K. Miller. 1990. WordNet: An online
lexical database. Int. J. Lexicograph. 3, 4, pp. 235-244.
27. Using computers in linguistics: a practical guide by John Lawler, Helen Aristar Dry 198
ISBN 0-415-16792-2 page 209
28. Naive semantics for natural language understanding by Kathleen Dahlgren 1988 ISBN 0-
89838-287-4
29. Stochastically-based semantic analysis by Wolfgang Minker, Alex Waibel, Joseph Mariani
1999 ISBN 0-7923-8571-3
30. Pragmatics and natural language understanding by Georgia M. Green 1996 ISBN 0-8058-
2166-X
31. Wong, Yuk Wah, and Raymond J. Mooney. "Learning for semantic parsing with statistical
machine translation (http://www.aclweb.org/anthology/N06-1056)." Proceedings of the main
conference on Human Language Technology Conference of the North American Chapter of
the Association of Computational Linguistics. Association for Computational Linguistics,
2006.
32. Natural Language Processing Prolog Programmers by M. Covington, 1994 ISBN 0-13-
629478-2
33. Natural language processing in Prolog by Gerald Gazdar, Christopher S. Mellish 1989
ISBN 0-201-18053-7
34. Understanding language understanding by Ashwin Ram, Kenneth Moorman 1999 ISBN 0-
262-18192-4 page 111
35. Formal aspects of context by Pierre Bonzon et al 2000 ISBN 0-7923-6350-7
36. Programming with Natural Language Is Actually Going to Work—Wolfram Blog (http://blog.w
olfram.com/2010/11/16/programming-with-natural-language-is-actually-going-to-work/)
37. Van Valin, Jr, Robert D. "From NLP to NLU" (http://www.isi.hhu.de/fileadmin/redaktion/Oeffe
ntliche_Medien/Fakultaeten/Philosophische_Fakultaet/Sprache_und_Information/Van_Valin
_From_NLP_to_NLU.pdf) (PDF).
38. Ball, John. "multi-lingual NLU by Pat Inc" (http://pat.ai/). Pat.ai.

Retrieved from "https://en.wikipedia.org/w/index.php?title=Natural-language_understanding&oldid=1144845161"