The Nature of Systematicity in Natural Language 27

The Nature of Systematicity in Natural Language 27 October, 2004 Blutner & Spenader KNAW-Colloquium

Systematicity • Introduced in Fodor & Pylyshyn (1988): – A feature of cognition, inferential coherence – ``The ability to produce/understand some sentences is intrinsically connected to the ability to produce/understand certain others. ‘’ (p. 37) – Fodor & Pylyshyn give no clear definition (see van Gelder) and only two examples 27 October, 2004 Blutner & Spenader KNAW-Colloquium 2

The operationalization of systematicity • When an agent understands the sentence Peter likes Maria, she understands the sentence Maria likes Peter as well. • When an agent understands the expressions brown triangle and black square, she understands the expressions brown square and black triangle as well. 27 October, 2004 Blutner & Spenader KNAW-Colloquium 3

Outline • What is systematicity? – and to what degree is our understanding systematic • Does compositionality entail systematicity? – No, because • • • context-dependence of meaning intrusion of encyclopedic knowledge on language understanding What are appropriate models of natural language understanding if systematicity is taken into account ? 27 October, 2004 Blutner & Spenader KNAW-Colloquium 4

Is our understanding systematic ? (1) Do all who understand good writer and bad teacher also understand bad writer and good teacher? (2) Do all who understand within an hour and without a watch also understand within a watch and without an hour? (Szabó 2004). (3) Does a child who understands Mom drives me to kindergarten also understand I drive mom to kindergarten ? 27 October, 2004 Blutner & Spenader KNAW-Colloquium 5

Understanding not Systematic • Language understanding is not completely systematic. – This is because the world is not entirely systematic – A. Clark (1996) ``Instead of treating [systematicity] as a property to be directly induced by a canny choice of basic architecture, it may be fruitful to try to treat it as intrinsic to the knowledge we want a system to acquire. • A good model of language – explains the systematic patterns of our language understanding – gives an account of when and why we find patterns that are not really predictable. 27 October, 2004 Blutner & Spenader KNAW-Colloquium 6

Does compositionality entail systematicity? • Compositionality: The meaning of a compound expression is determined by the meanings of its constituents and the syntactic way these constituents are combined. • Systematicity: there are predictable patterns among the sentences we understand. 27 October, 2004 Blutner & Spenader KNAW-Colloquium 7

The present position • Compositionality does not entail systematicity • Systematicity is possible without compositionality 27 October, 2004 Blutner & Spenader KNAW-Colloquium 8

How to derive systematicity? • Fodor & Pylyshyn: – in the presented case the systematicity of linguistic competence derives from the fact that the syntactic operation of modification relates to the semantic operation of conjunction (or intersection) 27 October, 2004 Blutner & Spenader KNAW-Colloquium 9

Understanding NL Expressions • According to the classical view, the phrase 'understanding natural language expressions' means more than just fixing a denotation. • We 'understand' a phrase as brown triangle if (i) we are able to grasp the corresponding conceptual representation (brown triangle) (ii) the corresponding conceptual components have an known truth functional content (iii) With the help of the involved logical operators it is possible to determine the truth conditions of the whole phrase under discussion 27 October, 2004 Blutner & Spenader KNAW-Colloquium 10

Sytematicity Clauses: A sample “derivation” • • • understanding the expressions brown triangle and black square deriving brown triangle and black square extracting the lexicon entries brown, black, triangle, square. calculating the corresponding conceptual representations for brown square and black triangle : brown square and black triangle can get the truth-conditional impact of these forms we understand the expressions brown square and black triangle 27 October, 2004 Blutner & Spenader KNAW-Colloquium 11

But does this actually work? • This proposal doesn’t work for even the simplest of adjective-noun combinations • Intersection doesn’t give you the systematic inferences that you would expect or would desire 27 October, 2004 Blutner & Spenader KNAW-Colloquium 12

Why compositionality fails to explain systematicity 27 October, 2004 Blutner & Spenader KNAW-Colloquium 13

Context dependence • In his “Grundlagen der Mathematik” Frege (1884) noticed the context-dependence of words (and took this as a argument against compositionality in NL) “One should ask for the meaning of a word only in the context of a sentence, and not in isolation“ • There are compositional models of semantics that take context dependence into account • We can use the phenomenon of contextdependence for construing an argument against deriving systematicity from compositionality. 27 October, 2004 Blutner & Spenader KNAW-Colloquium 14

What is a white triangle? A The hite w gle n tria A Th e ack bl gle n ria t From P. Bosch (2002) Explaining semantic productivity. Paper presented at the Symposium on Logic and Creativity: Integrating Categorial Rules and Experience, Osnabrück. 27 October, 2004 Blutner & Spenader KNAW-Colloquium 15

Context Dependence? Does the proposition expressed by ‘black triangle’ depend on the contrast set? Answer 1: No. The contrast set is relevant only for choosing a good/appropriate/optimal referring expression PROBLEM: ? Triangle 1 is black and white (in one and the same respect) Answer 2: Yes. Context-dependency of utterance meaning is widely accepted and also applies here (Kaplan’s characters; Haas-Spohn’s ‘hidden indexicality’; e. g. in connection with natural kind terms like water; theories of underspecification, e. g. Carstons 2002; explicatures in relevance theory) 27 October, 2004 Blutner & Spenader KNAW-Colloquium 16

What is a red apple? (a) a red apple [red peel] (b) a sweet apple [sweet pulp] (c) a reddish grapefruit [reddish pulp] (d) a white room/ a white house [inside/outside] A red apple? No, it’s a green apple but it’s red on the inside 27 October, 2004 Blutner & Spenader KNAW-Colloquium 17

More examples • Quine (1960) was the first who noted the contrast between red apple (red on the outside) and pink grapefruit (pink on the inside). • In a similar vein, Lahav (1993) argues that an adjective such as brown doesn’t make a simple and fixed contribution to any composite expression in which it appears: In order for a cow to be brown most of its body’s surface should be brown, though not its udders, eyes, or internal organs. A brown crystal, on the other hand, needs to be brown both inside and outside. A brown book is brown if its cover, but not necessarily its inner pages, are mostly brown, while a newspaper is brown only if all its pages are brown. For a potato to be brown it needs to be brown only outside, . . . (Lahav 1993: 76). 27 October, 2004 Blutner & Spenader KNAW-Colloquium 18

Three consequences • Intersectivity, ||A(B)|| = ||A( )|| ||B||, doesn’t hold for most ‘absolute’ adjectives and Fodor & Pylyshyn's compositional analysis breaks down • Systematicity statements cannot be derived from compositionality if intersectivity fails • Encyclopedic knowledge is required to determine the truth conditional content of an utterance (explicature in Relevance Theory) • When an agent understands the expressions red apple (RED PEEL) and sweet grapefruit (SWEET PULP), then it's likely that she understands red grapefruit (RED PULP) and sweet apple (SWEET PULP) as well. 27 October, 2004 Blutner & Spenader KNAW-Colloquium 19

The Theoretical Part 27 October, 2004 Blutner & Spenader KNAW-Colloquium 20

Approach 1: The underspecification view • • Radical underspecification augmented with contextual enrichment small x small(x, N) * small terrier x [small(x, N) & terrier(x)] Analogously for red apple with place-holders for the relevant parts red x [part(Y, x) & red(Y)] red apple x [part(Y, x) & red(Y) & apple(x)] How to determine the proper values for N and Y, respectively? * with small(x, N) size(x) < N 27 October, 2004 Blutner & Spenader KNAW-Colloquium 21

A mechanism of contextual enrichment • The variables are specified in a way that maximizes the relevance of the corresponding question - • Small Terrier: Is a (randomly selected) terrier smaller than N? Red Apple: What color is part Y (of a randomly selected apple)? Probabilistic Theory of Relevance, see Robert van Rooy (2000): Comparing Questions and Answers: A bit of Logic, a bit of Language, and some bits of Information 27 October, 2004 Blutner & Spenader KNAW-Colloquium 22

Entropy of a question • The semantic value of a question Q is a partition {q 1, . . . , qn} of the domain Ω. • inf(q) = -log 2 prob(q) information = measure of surprise • Entropy of a question Q • The entropy of a question expresses our uncertainty about the answer. Good questions have high entropies 27 October, 2004 Blutner & Spenader KNAW-Colloquium 23

What is a black triangle? What color is the inner part? What color is the outer part? 27 October, 2004 E=1 E=0 E=1 Blutner & Spenader KNAW-Colloquium 24

What is a red apple? A red apple? What color is an apple? Q 1 What color is its peel? Q 2 What color is its pulp? E(Q 1) >> E(Q 2) Color differences between apples are expected for the peel and not for the pulp. Therefore, the presented apple is considered as a green apple (inside red) and not as a red apple (outside green). This can change if we update our probability distribution. 27 October, 2004 Blutner & Spenader KNAW-Colloquium 25

Problems with the underspecification view • Requires rather clumsy lexical entries • How much of the peel of an apple has to be red in order to call it a red peel? • This theory does not really clarify how the border line between the underspecified representation and the contextual enrichment is ever to be determined 27 October, 2004 Blutner & Spenader KNAW-Colloquium 26

Approach 2: Adnominal functors • Take Montague (1970) as its starting-point and take adjectives as adnominal functors. • red(X) means roughly the property – (a) of having a red inner volume if X denotes fruits only the inside of which is edible – (b) of having a red surface if X denotes fruits with edible outside – (c) of having a functional part that is red if X denotes tools, … 27 October, 2004 Blutner & Spenader KNAW-Colloquium 27

Montague and systematicity clauses • We cannot derive the intended systematicity clauses if we realize compositionality via adnominal functors! • f(a) = p, g(b) = q • • • f(b) = ? , g(a) = ? Compositionality is often very simple to realize if (higher order) functions are introduced In many cases, this idea realizes generalizations to the worst case Unfortunately, we cannot derive interesting systematicity clauses from this style of compositionality! 27 October, 2004 Blutner & Spenader KNAW-Colloquium 28

Constraining the models • • Additional constraints are required that restrict the set of possible models I will assume that these constraints can be extracted from a Bayesian picture of the mental encyclopedia This is not so different from assuming a system of violable constraints (ranked defaults instead of inviolable meaning postulates) Systematicity mainly results from these additional constraints! 27 October, 2004 Blutner & Spenader KNAW-Colloquium 29

Approach 3: Connectionst model of adjectival modification* • • Overcoming the gap between compositionality and systematicity Modeling both the truth-functional aspects of adjectival modification and the typicality effects (Kamp & Partee 1995) Connectionist variant of the selective modification model of Smith et al. (1988) It shares with this model the (localist) attributevalue representation for the prototypes (apple, grapefruit, …) and for the relevant instances. * From Blutner, Hendriks, de Hoop, & Schwartz (to appear): When compositionality fails to predict systematicity 27 October, 2004 Blutner & Spenader KNAW-Colloquium 30

The simplified model Adjective Color Taste Noun Form Conceptual Layer 27 October, 2004 Blutner & Spenader KNAW-Colloquium 31

Conclusions (1) Systematicity of natural language understanding is intimately related to knowledge about the world (2) Systematicity clauses have to reflect this reality (3) A compositional representational system doesn’t give you systematicity of understanding because context-dependency of lexical meaning and world-knowledge both affect understanding 27 October, 2004 Blutner & Spenader KNAW-Colloquium 32

Conclusions cont. (4) If lexical principles like “intersectivity” fail (because of context dependency) systematicity clauses can no longer be derived from compositionality (5) A Bayesian and/or connectionist picture of the mental encyclopedia is the key for deriving systematicity clauses! (6) The principle of compositionality of meaning may be interesting not as part of cognitive architecture, but as a consequence of evolutionary learning (e. g. Kirby). 27 October, 2004 Blutner & Spenader KNAW-Colloquium 33

Thank you for your attention 27 October, 2004 Blutner & Spenader KNAW-Colloquium 34

. . . other problems with adjectival modification Typicality effects The problems of typicality don't relate to the truth-conditional impact of adjectival modification. However, a good model should account for both effects: the truth-conditional peculiarities of adjectival modification and the typicality effects. 27 October, 2004 Blutner & Spenader KNAW-Colloquium 35

The red hair problem • • Quine (1960): the different colors typically denoted (strong preference) by red in red apple and red hair In Japanese, aka-zatoo 'brown sugar' (lit. 'red sugar') comes in the same range of colors as shira-miso, lit. 'white bean paste‘ è the actual color value deviates in a systematic way from the prototypical color value that can be assigned to the color adjective in isolation – in dependency on the conceptual properties of the modified noun 27 October, 2004 Blutner & Spenader KNAW-Colloquium 36

The pet fish problem • • Goldfish is a poorish example of a fish, and a poorish example of a pet, but it's quite a good example of a pet fish (the conjunction effect of typicality: cx(A&B) > cx(B)) In case of "incompatible conjunctions" such as pet fish, striped apple or brown apple the conjunction effect is greater than in "compatible conjunctions“ (red apple). 27 October, 2004 Blutner & Spenader KNAW-Colloquium 37

Prototype effects • Using Tversky's (1977) contrast rule (formulated for activation vectors) sim(s, t) = i min(si, ti) i |si ti| • sim(sred apple, t 1) > sim(sapple, t 1) • sim(sbrown apple, t 2) sim(sapple, t 2) > sim(sred apple, t 1) sim(sapple, t 1) 27 October, 2004 Blutner & Spenader KNAW-Colloquium 38

Next steps • • Extend the model by distinguish different parts of the fruits (inside and outside) Since without further learning the model starts with a uniform weight matrix, we expect that the 'neutral' force of modification affects all parts uniformly After learning, that the color of the outside of fruits is more discriminating than the color of its inside, we expect that the learning mechanism correctly reproduces the expected sort of modification Similarities to Zwarts' (2003) "strongest meaning" model which starts with an initial default hypothesis which is subsequentially modified if more encyclopedic knowledge comes in. 27 October, 2004 Blutner & Spenader KNAW-Colloquium 39