Dependency Representations Grammars Folded Structures among Other Things

Dependency Representations, Grammars, Folded Structures among Other Things! Aravind K Joshi University of Pennsylvania Philadelphia USA DEPLING, 2013 August 28 2013 Charles University, Prague, Czech Republic

Outline • How complex dependencies can be? - Projective vs Nonprojective - Representations at different levels • Direct representations or via formal grammars • Empirical studies: What dependencies appear in annotated corpora and not just how often? • Dependencies as Folded Structures!!

Complexity of Dependencies • What do we know from the formal side? • Mildly Context Sensitive Languages (MCSL) -- includes Context Free Languages (CFL) -- Limited crossing dependencies -- Polynomially parsable

Complexity of Dependencies • Although TAG’s (TAL’s) are in MCSL, they are much more restricted! • Although TAG’s support crossing dependencies, these crossing dependencies are well-nested (nested dependencies of context-free languages are well-nested—Pumping Lemma!) Proposed in Joshi et al. 1985 but proved only in 2010 by Kanazwa! July 16 2010

Complexity of Dependencies • The language MIX, proposed by Bach (1982) as an Extreme Case of Scrambling! • Bach Language, MIX = { w| for each n, w is a string containing n a’s, n b’s, and n c’s, in any order} Treating, each w as a ‘scrambled’ version of n clauses, each clause containing one a, one b, and one c. • MIX is thus an extreme case of non-projectivity! July 16 2010

MIX • Joshi (1985) suggested that TAG’s and their variants introduced for linguistic descriptions should not be able to generate MIX, thereby excluding it from the class of Mildly Context Sensitive Languages (MCSL) • Many attempts to prove this conjecture did not succeed, until very recently! • Kanazawa and Sylvati finally proved this conjecture in 2012 (paper presented at ACL 2012)! July 16 2010

Complexity of Dependencies • Varieties of TAG grammars all weakly equivalent to TAG but capable of providing structural descriptions going beyond standard TAG -- Tree Local Multicomponent TAG (MCTAG) -- Very Limited use of Set Local MCTAG -- Adequate for Scrambling and Clitic Climbing constructions (Joint work with Joan Chen Main and Tonia Bleam, 2011, 2012)

SINGLE TREE or SETS of TREES requiring skilled tree surgeries to force a single tree over a sentence • • • Parentheticals Epithets Displaced adjectives, PP’s, etc. Right node raising Extraposition from NP Sentential relatives Coordinations

One tree covering the whole sentence W 1 W 2 W 3 W 4 W 5 W 6 • Single tree rooted in one root node • Every word is covered • All connections between the nodes are in the same dimension

Parentheticals Mary, John thinks, will win the race Arterial Roots Mary, John thinks, will win the race John thinks is attached to the root node of the Mary will win the race tree in an orthogonal dimension, reflecting the different semantic nature of this attachment

Epithets I finished the damn book Arterial Roots I finished the damn book is attached to the root node of the I finished the book tree an orthogonal dimension, reflecting the different semantic nature of this attachment damn

More examples • Extraposition from NP: The gardener finally came, who had the keys • Misplaced adjectives: An occasional sailor walked by • Sentential relatives: John believes* Mary will finish her dissertation this year, which no one expected her to do * This example is from Bonnie Webber.

Some benefits for not insisting on a single tree • Not going for a single tree may ease the burden on the annotators • Sometimes syntax does more work than necessary! Very often at the discourse annotation stage some work done by syntax has to be undone. -- Syntax should have annotated the sentence with two chunks linked in an orthogonal dimension ** It is only at the discourse annotation stage the final decision of the attachment can be made -- Striking and very frequent examples of this situation arise in ATTRIBUTION

Ø There have been no orders for the Cray-3 so far, though the company says it is talking with several prospects. ü Discourse semantics: contrary-to-expectation relation between “there being no orders for the Cray-3” and “there being a possibility of some prospects”. Sentence semantics: contrary-to-expectation relation between “there being no orders for the Cray-3” and “the company saying something”. S NP VP VP There have been no Orders for the Cray-3 SBAR-ADV IN though S NP the company VP V says Discourse arguments Syntactic arguments S it is talking With several prospects

• Attribution cannot always be excluded by default Advocates said the 90 -cent-an-hour rise, to $4. 25 an hour by April 1991, is too small for the working poor, while opponents argued that the increase will still hurt small business and cost many thousands of jobs In this example, the attributing phrases stay with the arguments of the connective while This decision can only be made at the discourse level At the sentence level two trees covering the sentence with the two trees connected in an orthogonal dimension would have been the best decision!

Annotation Overview: Attribution in WSJ 34% of discourse relations are attributed to an agent other than the writer.

Types of Dependencies • Word to Word John loves mangoes John bought the house Predicate argument relation? 17

Types of Dependencies Word to Phrase John bought the house Predicate argument relation? 18

Types of Dependencies Phrase to Word John took a walk 19

Types of Dependencies Phrase to Phrase The old man took a walk 20

Types of Dependencies How much of the phrase to be included in the argument? By convention (? ) we take the maximal phrase. John bought [the house next door which was on sale for over a year] the house next door which was on sale for over a year What about the minimal phrase that is sufficient to identify 21 the referent in the context (discourse context, for example)?

Ø There have been no orders for the Cray-3 so far, though the company says it is talking with several prospects. ü Discourse semantics: contrary-to-expectation relation between “there being no orders for the Cray-3” and “there being a possibility of some prospects”. Sentence semantics: contrary-to-expectation relation between “there being no orders for the Cray-3” and “the company saying something”. S NP VP VP There have been no Orders for the Cray-3 SBAR-ADV IN though S NP the company VP V says Discourse arguments Syntactic arguments S it is talking With several prospects 22

Ø Although takeover experts said they doubted Mr. Steinberg will make a bid by himself, the application by his Reliance Group Holdings Inc. could signal his interest in helping revive a failed labor-management bid. ü Discourse semantics: contrary-to-expectation relation between “Mr. Steinberg not making a bid by himself” and “the RGH application signaling his bidding interest”. Sentence semantics: contrary-to-expectation relation between “experts saying something” and “the RGH application signaling Mr. Steinberg’s bidding interest”. S SBAR-ADV IN Although NP-SBJ S the application by his RGH Inc. NP-SBJ VP takeover experts VBD SBAR said NP-SBJ VP MD could VP VB NP signal VP they VBD doubted SBAR Mr. Steinberg will make a bid by himself his interest in helping revive a failed labormanagement bid 23

• Attribution cannot always be excluded by default Ø Advocates said the 90 -cent-an-hour rise, to $4. 25 an hour by April 1991, is too small for the working poor, while opponents argued that the increase will still hurt small business and cost many thousands of jobs. 24

Do we want a single tree over a sentence? • There are many constructions in language that suggest that the single tree hypothesis may be wrong -- Parentheticals, supplements, sentential relatives, among others are problematic for the single tree hypothesis Mary, John thinks, will win the election (John thinks is attached to the S node medially but it has scope over Mary will win the election) 25

John heard that Mary finally finished her dissertation, which no one ever expected her to do so ( (1) John heard that and (2) which no one ever expected her to do both have scope over (3) Mary finally finished her dissertation. Both (1) and (2) are attached to the root node S but neither (1) nor (2) have scope over the other) 26

Alternative Lexicalization (Alt. Lex) A discourse relation is inferred between two sentences which do not contain an Explicit connective, but insertion of an Implicit connective leads to redundancy. This is because the relation is alternatively lexicalized by some non-connective expression: Ø Under a post-1987 crash reform, the Chicago Mercantile Exchange wouldn’t permit the December S&P futures to fall further than 12 points for a half hour. Alt. Lex = (consequence) That caused a brief period of panic seeling of stocks on the Big Board. 27

Alt. Lex expressions often do not correspond to syntactic constituencies. Under a post-1987 crash reform, the Chicago Mercantile Exchange wouldn’t permit the December S&P futures to fall further than 12 points for a half hour. Alt. Lex = (consequence) That caused a brief period of panic selling of stocks on the Big Board. S NP-SBJ VP DT VBD That caused DT a brief period PP-LOC of panic selling…. . 28

Syntactic Structures as Folded Structures Analogous to Secondary or Tertiary Structures of Biomolecules

Biological Sequences • DNA, RNA, PROTEIN Sequences -- DNA and RNA: sequences of four nucleotides -- A, C, G, and T or A, C, G, and U -- Matching Pairs: A, T(U) and C, G -- Proteins: Sequences of twenty amino acids cis 630 -4 -10: 30

RNA secondary structure cis 630 -4 -10: 31

RNA secondary structure: Pseudoknots cis 630 -4 -10: 32

Dependencies as Folded Structures John has eaten apples eaten John has cis 630 -4 -10: 33

Dependencies as Folded Structures John has eaten apples John has eat en apples eat John has en cis 630 -4 -10: 34

Subject Relatives The cat NP 1 V 2 that NP 1 chased the rat V 1 NP 2 that V 1 fled V 2 NP 2 cis 630 -4 -10: 35

Object Relatives The cat NP 1 that the dog chased fled NP 2 V 1 Goes into another plane and comes out V 1 NP 1 that V 2 NP 2 Object Relatives are more complex than Subject Relatives, even at the first level. cis 630 -4 -10: 36

Crossing Versus Nested Dependencies Crossing (1) NP 1 NP 2 V 1 V 2 (2) NP 1 NP 2 NP 3 V 1 V 2 V 3 Both (1) and (2) can be folded in one plane! Nested (1’) NP 1 NP 2 V 1 (2’) NP 1 NP 2 NP 3 V 2 V 1 (1’) can be folded in one plane. (2’) cannot be folded in one plane. Beyond 2 levels of embedding this difference disappears!! cf Bach and Marslen Wilson (1985) cis 630 -4 -10: 37

RNA secondary structure: Pseudoknots cis 630 -4 -10: 38

Pseudoknots in Linguistic Structures N 1 N 2 N 3 V 2 V 1 Move N 2 N 3 V 2 to the right of V 1 and then Move N 2 N 3 back N 1 V 1 N 2 N 3 V 2 V 3 N 1 N 2 N 3 V 1 V 3 V 2 cis 630 -4 -10: 39

Pseudoknots In Linguistic Structures N 1 N 2 N 3 V 1 V 3 V 2 Remnant Extraposition N 1 V 1 N 2 N 3 V 2 V 3 FOLDED STRUCTURE AS THE SYNTACTIC STRUCTURE!! cis 630 -4 -10: 40

Folded Structures as Syntactic Structures When Subject Relatives and Object Relatives are represented as Folded Structures, Object Relatives are more costly than Subject Relatives -- even at the first level of embedding! Object Relatives require going out of the plane and coming back up as in the case of parallel strands! Optimization with respect to Folded Structures !! cis 630 -4 -10: 41