Deriving General World Knowledge from Texts and Taxonomies,
Featuring the KNEXT System

Participants:

Len Schubert
Department of Computer Science
Room 733, Computer Studies Building
University of Rochester
Greg Carlson
Department of Linguistics
University of Rochester
Web home page: http://www.ling.rochester.edu/people/carlson/carlson.html
Henry Kyburg
Departments of Computer Science and Philosophy
University of Rochester
Web home page: http://www.cs.rochester.edu/u/www/u/kyburg/
Phil Michalak
Department of Computer Science
University of Rochester
Web home page: http://www.cs.rochester.edu/~michalak/
Fabrizio Morbini
Department of Computer Science
University of Rochester
Web home page: http://www.cs.rochester.edu/~morbini/
Benjamin Van Durme
Department of Computer Science
University of Rochester
Web home page: http://www.cs.rochester.edu/~vandurme/
Saurabh Deshpande
Department of Computer Science
University of Rochester
Web home page: http://www.cs.rochester.edu/~saurabh/
Jivko Sinapov
Department of Computer Science
University of Iowa

Past participants:

Matthew Tong
Department of Computer Science
Univ. of California at San Diego
http://www.cs.ucsd.edu/csepeople/graduatestudenthomepages.html
David Ahn
Informatics Institute
University of Amsterdam
Kruislaan 403
1098 SJ Amsterdam
The Netherlands
Web home page: http://staff.science.uva.nl/~ahn/
Aaron Kaplan
Xerox Research Centre Europe (XRCE)
6, chemin de Maupertuis
38240 Meylan, France
Web home page: http://www.xrce.xerox.com/people/akaplan/

Summary

We think that there is a largely untapped source of general knowledge in texts, lying at a level beneath the explicit assertional content. This knowledge consists of relationships implied to be possible in the world, or, under certain conditions, implied to be normal or commonplace in the world. For instance, the sentence "He entered the house through its open door" suggests that it is possible for a person (or at least a male) to enter a house, that houses have doors, that doors can be open, etc. The goal of the present work is to derive such general world knowledge (initially, from Penn Treebank corpora) in two stages: first, we derive general "possibilistic" propositions from noun phrases and clauses; then we try to derive stronger generalizations, based on the nature and statistical distribution of the possibilistic claims obtained in the first phase.

Our initiative differs from standard knowledge extraction work in both its aims and its methodology. We are attempting to derive a broad range of general relationships from texts, rather than some predetermined specific kinds of facts; and we are using general phrase structure coupled with compositional interpretive rules to obtain general propositional information, rather than employing specialized extraction patterns targeted at specific relationships. Our long-range goal is to use the derived knowledge as part of a KB supporting language understanding and commonsense reasoning.

We have reached a stage in our work where we are able to extract large numbers of general propositions from Treebank corpora using a program we are calling KNEXT (Knowledge EXtraction from Text), and have been able to assess, using multiple human judges, the prospects for arriving at useful world knowledge by this method, and the limitations of the approach.

An example of a complete and unedited output from KNEXT for a sentence from the Brown corpus is the following (omitting the input bracketing):

  (BLANCHE KNEW 0 SOMETHING MUST BE CAUSING STANLEY 'S NEW , STRANGE
   BEHAVIOR BUT SHE NEVER ONCE CONNECTED IT WITH KITTI WALKER .)
  
  OUTPUT (IN ENGLISH, FOLLOWED BY UNDERLYING LOGICAL FORMS):
  
   A FEMALE-INDIVIDUAL MAY KNOW A PROPOSITION.
   SOMETHING MAY CAUSE A BEHAVIOR.
   A MALE-INDIVIDUAL MAY HAVE A BEHAVIOR.
   A BEHAVIOR CAN BE NEW.
   A BEHAVIOR CAN BE STRANGE.
   A FEMALE-INDIVIDUAL MAY CONNECT A THING-REFERRED-TO WITH A FEMALE-INDIVIDUAL.
  
  ((:I (:Q DET FEMALE-INDIVIDUAL) KNOW[V] (:Q DET PROPOS))
   (:I (:F K SOMETHING[N]) CAUSE[V] (:Q THE BEHAVIOR[N]))
   (:I (:Q DET MALE-INDIVIDUAL) HAVE[V] (:Q DET BEHAVIOR[N]))
   (:I (:Q DET BEHAVIOR[N]) NEW[A]) (:I (:Q DET BEHAVIOR[N]) STRANGE[A])
   (:I (:Q DET FEMALE-INDIVIDUAL) CONNECT[V] (:Q DET THING-REFERRED-TO)
    (:P WITH[P] (:Q DET FEMALE-INDIVIDUAL))))
  

In this way, tens of thousands of propositions have been extracted from the Brown corpus (a 1,000,000-word segment of the Penn Treebank), and of these, approximately two thirds are judged to comprise "reasonable claims about the world" by human judges (while the rest are judged to be too vague, ambiguous, incomplete, or faulty in some other way to count as reasonable claims). We are currently working on the much larger (200,000,000-word) British National Corpus (BNC), using a parsed version made available to us by Dan Gildea, who applied Michael Collins' treebank parser to the corpus. We are in the process of filtering and assessing the several million propositions obtained by KNEXT from this parsed corpus. We are also attempting to abstract strengthened propositions from sets of related extracted propositions, and building a browser that will give some oversight over the extracted propositions by enabling focused retrieval. We are also working on new versions of KNEXT based on linguistically better-founded, more informative parses and more detailed logical interpretations of the parses obtained. In particular, our goal is to extract causal relations from sentences involving adverbial modifiers, as in "The wounded man died because of blood loss", or "Applying the defibrillator, the paramedic revived the heart attack victim".

A related effort is concerned with the development of a large lexical semantics knowledge base by manual and automated means, as another essential component of a general knowledge base for language understanding and commonsense inference, and as a tool for disambiguating propositions extracted by KNEXT. For instance, we would like to distinguish different senses of HAVE in propositions such as "A person may have an arm", "A person may have an accident", "A person may have a car", etc. The methods we are currently exploring include the use of Google queries to determine semantic properties of lexical nominals such as whether they are mass or count nouns ("gold" vs. "ring"), whether they typically refer to parts or wholes ("steering-wheel" vs. "car"), whether they refer to events or nonevents ("accident" vs. "president"), etc. For example, numerous Google hits for "much gold" (75,600), and relatively few hits for "much ring" (623) (normalized by the occurrence frequencies for "gold" and "ring") tend to suggest that "gold" is a mass noun while "ring" is a count noun.

In past work, we also investigated the potential usefulness of WordNet taxonomies. In particular, we attempted to determine how reliably hyponym-hypernym links between synsets can be viewed as providing subtype relations between predicates (for instance, that a marimba is a percussion instrument, or that a scythe is an edge tool) and how reliably hyponyms of the same synset can be viewed as mutually exclusive (for instance, that no percussion instrument is both a marimba and a triangle, or that no edge tool is both a scythe and a shear). We found that roughly 2 out of 3 of the hypernym links for nominals correspond to a true subtype relation, and a slightly higher percentage of ``sibling" hyponyms are truly exclusive. After noting some apparent reasons for failure of these properties, we attempted to improve the precision of the information extraction process. At the time, we found that the causes of were too diverse to enable large improvement by any automated means. Our new work on lexical semantics (above) suggests, however, that better lexical sematic feature annotations (esp. a reliable mass/count distinction and distinctions between deverbal and deadjectival nominals) will allow identification of many potentially erroneous supertypr-subtype relations. Even so, a significant residual error rate for such extracted information is probably unavoidable. Hence any practical application making use of this information would have to be error-tolerant, perhaps using probabilistic weights for subtype and exclusion relations.

The fact that knowledge extracted from texts or from lexicons is bound to remain somewhat "risky" points to the need for inference methods that make effective use of such knowledge. A number of our theoretical investigations are aimed in that direction. These include the development of a new algebraic characterization of probabilities in Bayesian networks, as a step towards a quantified logic allowing Bayes network-like causal inference, and the development of an evidence-based nonmonotonic logic of risky knowledge.

Finally, another aspect of the work concerns the semantics and pragmatics of generic sentences, such as "Elementary school children in Rochester are usually bused to school". Generic sentences express general facts, and it is just these sorts of facts we are attempting to derive from texts. A key problem is that of filling in implicit information about the situations, events, or entities quantified over. For example, the preceding sentence seems to quantify (using adverbial quantifier "usually") over situations where elementary school children in Rochester are going to school. We have developed theories (and some preliminary algorithms) for fleshing out the descriptions of such situations by accommodating the presuppositions of the matrix clause.

Acknowledgement The work on this project was previously supported by the National Science Foundation under grant IIS-0082928, "ITR: Mining Text for General World Knowledge", 9/1/2000 - 8/31/2003, and is currently supported by NSF grant IIS-0328849, "IIS: Deriving General World Knowledge from Texts by Abstraction of Logical Forms", 9/1/2003 - 8/31/2006.

Some relevant publications