Textual Entailment: What Does “If” Mean?

Posted on January 14, 2007 by Peter Turney

Recognizing Textual Entailment has become a popular task in Computational Linguistics, due to its relevance for Question Answering, Information Retrieval, Information Extraction, and Summarization. The task involves pairs of Text-Hypothesis sentences, such as the following:

Text: “Cavern Club sessions paid the Beatles £15 evenings and £5 lunchtime.”

Hypothesis: “The Beatles perform at Cavern Club at lunchtime.”

The task is to develop algorithms that can determine whether the Text sentence entails the Hypothesis sentence. That is, if Text is true, does it follow that Hypothesis is true? This raises the question, what does if mean?

In the RTE Challenge, human annotators label each Text-Hypothesis pair as “true” or “false”. These labels are the gold standard by which the competing algorithms are evaluated. This means RTE requires a descriptive theory of the meaning of if, rather than a normative theory. That is, to do well in this challenge, an algorithm must be based on how humans actually understand if, and not on how humans should understand if.

Propositional calculus provides a normative theory of if, which is concisely expressed in the truth table for material implication. One problem with using propositional calculus for RTE is that the Wason Selection Task shows that human understanding of if is not captured by material implication. That is, propositional calculus is not a good descriptive theory of if. Logicians have long had objections to material implication, which is why they developed relevance logic.

Conditional probability has been proposed as an alternative theory of if and it has had some degree of success in explaining the observations of the Wason Selection Task. However, Stenning and Lambalgen‘s paper, Semantics as a Foundation for Psychology: A Case Study of Wason’s Selection Task, makes a detailed and persuasive argument that we currently have no satisfactory descriptive theory of if. They write, “The purpose of this paper is to try to draw the attention of logicians and semanticists to this area, since we believe that empirical investigation of the cognitive processes involved could benefit from semantic analyses.” I believe that some benefit may also flow in the other direction, from the empirical investigation of cognitive processes to the field of semantics.