Cognition, Computers, and Car Bombs:  How Yale
Prepared Me for the 90’s
Wendy G. Lehnert
Department of Computer Science
University of Massachusetts
Amherst, MA 01003
From Yeshiva to Yale (1974)
From the East Side in midtown Manhattan, it was a brisk 20-minute walk going west to Eighth Avenue, another 30 minutes going north on the A train to 182nd Street, and then a final 10-minute walk going east to get to the Belfer Graduate School of Science at Yeshiva University. I made that trip every day for two years as a graduate student in mathematics. The math department was housed in a modern high-rise that stood out among the older and less majestic buildings of Washington Heights. Within that seemingly secular structure, each office door frame was uniformly adorned with a small white plastic mezuzah, courtesy of the university.
I thought a lot about what I was doing with my life during those subway rides. It was probably on the subway that I realized I was more interested in how mathematicians manage to invent mathematics than I was in the actual mathematics itself. I mentioned this to one of my professors, and his reaction was polite but pointed. Academic math departments had no room for dilettantes. Anyone who was primarily interested in the cognitive processes of mathematicians did not belong in mathematics and was well-advised to pursue those interests elsewhere. Given the abysmal state of the job market for PhDs in mathematics, I took the hint and set out to broaden my horizons.
One day I was browsing in the McGraw-Hill bookstore, and I stumbled across a collection of early writings on artificial intelligence (Feigenbaum and Feldman 1963). It was here that I learned about a community of people who were trying to unravel the mysteries of human cognition by playing around with computers. This seemed a lot more interesting than Riemannian manifolds and Hausdorff spaces, or maybe I was just getting tired of all that time on the subway. One way or another, I decided to apply to a graduate program in computer science just in case there was some stronger connection between FORTRAN and human cognition than I had previously suspected. When Yale accepted me, I decided to throw all caution to the wind and trust the admissions committee. I packed up my basenji and set out for Yale in the summer of 1974 with a sense of grand adventure. I was moving toward light and truth, and my very first full screen text editor.
As luck would have it, Professor Roger Schank, a specialist in artificial intelligence (AI) from Stanford, was also moving to Yale that same summer. Unlike me, Schank knew quite well what to expect in New Haven. He was moving to Yale so he could collaborate with a famous social psychologist, Robert Abelson, on models of human memory. Within a few short months, a fruitful collaboration between Schank and Abelson was underway. Schank was supporting a group of enthusiastic graduate students, and I was writing LISP code for a computer program that read stories and answered simple questions about those stories. 1
1LISP may not be significantly closer to human cognition than FORTRAN, but it does drive home the difference between number crunching and symbol crunching. Mainstream artificial intelligence operates on the assumption that intelligent information processing can be achieved through computational symbol manipulation.
I was amazed to discover how difficult it is to get a computer to understand even the simplest sentences, and I began to think about what it means for a human to understand a sentence. I wasnÕt particularly interested in the problems of vague or misleading language when what you heard isnÕt quite the same thing as what was said. I was more preoccupied with seemingly trivial sentences like ÒJohn gave Mary a book,Ó and the underlying mechanisms that enable us to understand that giving a
book is conceptually different from giving a kiss. Two things about this phenomenon seemed astonishing to me. First, it was remarkable that people ever managed to communicate anything at all with their sentences. And second, there appeared to be no body of expertise that could shed much light on the mental processes associated with this most mundane level of language comprehension.
Another Yale computer scientist, Alan Perlis (famous for his APL one-liners among other things), was rather adept at witty aphorisms. One of my favorites was this one: With computers, everything is possible and nothing is easy. While the first claim constitutes an article of faith, the second claim is readily apparent to anyone who has ever written a computer program. I believed without question that computers could be made to understand sentences. Even so, it was humbling to discover that the bland activities of John and Mary were somehow more elusive to me than highly abstract theorems of differential geometry and functional analysis. In fact, I was beginning to suspect that one might possibly devote an entire lifetime to John and Mary and the book without ever getting it quite right. It is true that John and Mary lack the intellectual cachet of high powered mathematics, but I no longer believed that my mathematician friends had a monopoly over all the hard problems.
Fast Forward (June 1991)
The place is the Naval Ocean Systems Center in San Diego. I am attending a relatively small, invitation-only meeting with one of my graduate students. The purpose of the meeting is to discuss the outcome of a rigorous performance evaluation in text extraction technologies. Fifteen laboratories have labored for some number of months (one person/year of effort, on average) to create computer systems that can comprehend news stories about terrorism. Each system has taken a rigorous test designed to assess its comprehension capabilities. This particular test consisted of 100 texts, previously unseen by any of the system developers, which were distributed to each of the participating laboratories along with strict testing procedures. Each system was required to (1) extract a database of essential facts from the texts without any human intervention, and (2) be graded against a hand-coded database containing all the correctly encoded facts. The scoring of the test results was conducted by yet another system (the scoring program) which was scrupulously precise and relentlessly thorough in its evaluations.
This unusual meeting is called MUC-3 (a.k.a. the Third Message Understanding Conference), and three university sites have participated in the evaluation along with 12 industry labs.  My student and I represent the University of Massachusetts at Amherst. Most of the people here have been involved with natural language processing for at least a decade or more. We no longer discuss how to tackle ÒJ
ohn gave Mary a book.Ó Now we debate different ways to measure recall and precision and overgeneration. We talk about spurious template counts, grey areas in the domain guidelines, and whether or not our training corpus of 1300 sample texts was large enough to provide an adequate training base. When we talk about specific sentences at all, we talk about real ones:
THE CAR BOMB WAS LEFT UNDER THE BRIDGE ON 68TH STREET AND 13TH
STREET WHERE IT EXPLODED YESTERDAY  AT APPROXIMATELY 1100,
KILLING MARIA JACINTA PULIDO, 42;  PILAR PULIDO, 19; A MINOR
REPORTEDLY KNOWN AS CARLOS; EFRAIN RINCON RODRIGUEZ, AND A POLICE
OFFICIAL WHO DIED AT THE POLICE CLINIC.
Figure 1:  The MUC Method for Evaluating Computational Text Analyzers
The purpose of the meeting is threefold. First, we are hoping to assess the state-of-the-art in natural language processing as it applies to information extraction tasks. Second, we would like to achieve so
me greater understanding about which approaches work well and which approaches are choking. Third, we are interested in the problem of technology evaluations and what it takes to get an objective assessment of our respective systems. Many of us bring hard-won years of experience and research to the table. We are curious to see what all that background will buy us. MUC-3 is an exciting meeting because it signifies a first attempt at a serious evaluation in natural language processing. Evaluations had been conducted prior to 1991 in speech recognition2, but nothing has been attempted in natural language processing until now.
Before we discuss the outcome of the evaluation, some observations about the MUC-3 meeting are in order. Most importantly,  the researchers who are here represent a broad spectrum of approaches to natural language processing. MUC-3 attracted formal linguists who concentrate on complicated sentence grammars, connectionists who specialize in models of neural networks, defense contractors who happily incorporate any idea that looks like it might work, and skilled academics who have based entire research careers on a fixed set of assumptions about the problem and its solutions. It is unusual to find such an eclectic gathering under one roof. The social dynamics of the MUC-3 meeting are an interesting topic in its own right.
On the one hand, we have 15 sites in apparent competition with one another. On the other hand, we h
ave 15 sites with a strong common bond. Each MUC-3 site knew all too well the trauma of preparing for MUC-3 and the uneasy prospect of offering up the outcome for public scrutiny. Like the survivors of some unspeakable disaster, we have gathered in San Diego to trade war stories, chuckle over the twisted humor that is peculiar to folks who have been spending a little too much time with their machines, and explore a newfound sense of common ground that wasnÕt there before. We all wanted to understand what made each of the systems work or not work. We all wanted to identify problem areas of high impact. And we all wanted to see where we each stood with respect to competing approaches. It was an intensely stimulating meeting.
Flash Back (1978)
This episode takes place at Tufts University in Boston. Roger Schank and Noam Chomsky have agreed to appear in a piece of academic theater casually known as The Great Debate. On ChomskyÕs side we have the considerable momentum of an intellectual framework that reshaped American linguistics throughout the 60s: a perspective on language that is theoretical, permeated with abstractions, and exceedingly careful to distinguish the theoretical aspects of language from empirical language phenomena. On SchankÕs side we have a computational perspective on language that emphasizes human memory models, inference generation, and the claim that meaning is the engine th
at drives linguistic communication. Chomsky proposes formalisms that address syntactic structures: from this perspective he attempts to delineate the innate nature of linguistic competence. Schank is interested in building systems that work: he makes pragmatic observations about what is needed to endow computers with human-like language facilities.
Chomsky rejects the computational perspective because it resists containment by an agreeable formalism.  Schank rejects ChomskyÕs quest for formalisms because the problem he wants to solve is big and messy and much harder than anything that any of the formalists are willing to tackle. As with all Great Debates, both sides are passionately convinced that the opposition is hopelessly deluded. There is no common ground. There is no room for compromise. There is no resolution.
2Speech recognition refers to the comprehension of spoken language, as opposed to natural language processing  which assumes input in the form of written language.
I wonÕt say who won The Great Debate. I wasnÕt there myself. But various manifestations of The Great Debate haunted much of SchankÕs academic life in one way or another throughout much of the 70Õs. As a graduate student in SchankÕs lab, I was thoroughly sensitized to a phenomena that is not unrelated to The Great Debate. It was a phenomenon associated with methodological styles. Simply put, some researchers are problem-driven and some researchers are technology-driven.
Problem-driven researchers start with a problem and look for a technology that can handle the problem. Sometimes nothing works very well and a new technology has to be invented. Technology-driven researchers start with a technology and look for a problem that the technology can handle. Sometimes nothing works very well and a new problem has to be invented. Both camps are equally dedicated and passionate about their principal alliance. Some of us fall in love with problems and some of us fall in love with technologies. Does a chicken lay eggs to get more chickens or do eggs make chickens to get more eggs?
As a student who was privileged to attend many research meetings with Bob Abelson, I learned that thought processes and personality traits often interact in predictable ways. Moreover, community standards and social needs are important variables in cognitive modeling. When you turn the lessons of social psychology back on to the scientific community, you discover that researchers, being just as human and social as anyone else, exhibit many predictive features that correlate with specific intellectual orientations. In particular, certain personality traits go hand and hand with certain styles of research. Schank and Abelson hit upon one such phenomenon along these lines and dubbed it the neats vs. the scruffies. These terms moved into the mainstream AI community during the early 80s, shortly after Abelson presented the phenomenon in a keynote address at the Annual Meeting of the Co
gnitive Science Society in 1981. Here are some selected excerpts from the accompanying paper in the proceedings:ÒThe study of the knowledge in a mental system tends toward both naturalism and phenomenology. The mind needs to represent what is out there in the real word, and it needs to manipulate it for particular purposes. But the world is messy, and purposes are manifold. Models of mind, therefore, can become garrulous and intractable as they become more and more realistic.
If oneÕs emphasis is on science more than on cognition, however, the canons of hard science dictate a strategy of the isolation of idealized subsystems which can be modeled with elegant productive formalisms. Clarity and precision are highly prized, even at the expense of common sense realism. To caricature this tendency with a phrases from John Tukey (1969), the motto of the narrow hard scientist is, ÒBe exactly wrong, rather than approximately rightÓ.
The one tendency points inside the mind, to see what might be there. The other points outside the mind, to some formal system which can be logically manipulated [Kintsch et al., 1981]. Neither camp grants the other a legitimate claim on cognitive science. One side says,ÒWhat you are doing may seem to be science, but itÕs got nothing to do with cognition.Ó The other side says, ÒWhat youÕre doing may seem to be about cognition, but itÕs got nothing to do with science.Ó
Superficially, it may seem that the trouble arises primarily because of the two-headed name cognitive science. I well remember discussions of possible names, even though I never liked Òcognitive scienceÓ, the alternatives were worse: abominations like ÒepistologyÓ or ÒrepresentonomyÓ.
But in any case, the conflict goes far deeper than the name itself. Indeed, the stylistic division is the same polarization that arises in all fields of science, as well as in art, in politics, in religion, in child rearing -- and in all spheres of human endeavor. Psychologist Silvan Tomkins (1965) characterizes this overriding conflict as that between characterologically left-wing and right-wing world views. The left-wing personality finds the sources of value and truth to lie within individuals, whose reactions to the world define what is important. The right-wing personality
asserts that all human behavior is to be understood and judged according to rules or norms which exist independent of human reaction. A similar distinction has been made by an unnamed but easily guessed colleague of mine, who claims that the major clashes in human affairs are between the ÒneatsÓ and the ÒscruffiesÓ. The primary concern of the neat is that things should be orderly and predictable while the scruffy seeks the rough-and-tumble of life as it comes ...
The fusion task is not easy. It is hard to neaten up a scruffy or scruffy up a neat. It is difficult to formaliz
e aspects of human thought that which are variable, disorderly, and seemingly irrational, or to build tightly principled models of realistic language processing in messy natural domains. Writings about cognitive science are beginning to show a recognition of the need for world-view unifications, but the signs of strain are clear ...
Linguists, by and large, are farther away from a cognitive science fusion that are the cognitive psychologists. The belief that formal semantic analysis will prove central to the study of human cognition suffers from the touching self-delusion that which is elegant must perforce be true and general. Intense study of quantification and truth conditions because they provide a convenient intersection of logic and language will not prove any more generally informative about the range of potential uses of language than the anthropological analysis of kinship terms told us about culture and language. On top of that, there is the highly restrictive tradition of defining the user of language as a redundant if not defective transducer of the information to be found in the linguistic corpus itself. There is no room in this tradition for the human as inventor and changer and social transmitter of linguistic forms, and of contents to which those forms refer. To try to understand cognition by a formal analysis of language seems to me like trying to understand baseball by an analysis of the physics of what happens when an idealized bat strikes an idealized baseball. One might learn a lot about possible traje
ctories of the ball, but there is no way in the world one could ever understand what is meant by a double play or a run or an inning, much less the concept of winning the World Series. These are human rule systems invented on top of the structural possibilities of linguistic forms. Once can never infer the rule systems from a study of the forms alone.
reaction to a book or an articleWell, now I have stated a strong preference against trying to move leftward from the right.
What about the other? What are the difficulties in starting our from the scruffy side and moving toward the neat? The obvious advantage is that one has the option of letting the problem areas itself, rather than the available methodology, guide us about what is important. The obstacle, of course, is that we may not know how to attack the important problems. More likely, we may think we know how to proceed, but other people may find our methods sloppy. We may have to face accusations of being ad hoc, and scientifically unprincipled, and other awful things.
(pp. 1-2 from [Abelson 81])
------
I periodically go back to this paper, about once every year or two, to think about AbelsonÕs observatio
ns in the context of my current research activities. I am always surprised to find new light and truth shining through with each subsequent reading. 3
The Ad Hoc Thing (1975)
This flashback takes place on the campus of the Massachusetts Institute of Technology. I am giving my first conference talk at TINLP (Theoretical Issues in Natural Language Processing). Schank and Abelson have been promoting the idea of scripts as a human memory structure and my talk describes work with scripts as well [Lehnert 1975]. MinskyÕs notion of a frame is also getting a lot of attention, and 3I am not the only one who still thinks about the neats and the scruffies.  Marvin Minsky recently published a paper called ÒLogical Versus Analogical or Symbolic Versus Connectionist or Neat Versus ScruffyÓ [Minsky 1991].

版权声明：本站内容均来自互联网，仅供演示用，请勿用于商业和其他非法用途。如果侵犯了您的权益请与我们联系QQ:729038198，我们将在24小时内删除。