Скачать презентацию INTRODUCCIÓ La major part de les aplicacions Скачать презентацию INTRODUCCIÓ La major part de les aplicacions

ba8355d2d265b32fbca530fb51295ac7.ppt

  • Количество слайдов: 97

INTRODUCCIÓ • La major part de les aplicacions interessants de llenguatge requeririen obtenir la INTRODUCCIÓ • La major part de les aplicacions interessants de llenguatge requeririen obtenir la representació del significat de les oracions.

Estructura predicat argument La estructura predicat argument descriu les relacions semàntiques que es donen Estructura predicat argument La estructura predicat argument descriu les relacions semàntiques que es donen entre les entitats que apareixen en la oració. -who does what to whom, -how, where, why?

Estructura predicat argument I eat sushi PRED: eat; ARG 1: I; ARG 2: sushi. Estructura predicat argument I eat sushi PRED: eat; ARG 1: I; ARG 2: sushi.

Un exemple més complex. • En frases complexes tenim més de una proposició. – Un exemple més complex. • En frases complexes tenim més de una proposició. – Mary loves the man who bougth the blue car – P 1: Mary loves the man. – P 2: The man bought the car. – P 3: blue car.

Perquè serveix la estructura sintàgmatica? • Per obtenir la estructura predicat argument es necessari Perquè serveix la estructura sintàgmatica? • Per obtenir la estructura predicat argument es necessari computar primer la estructura sintagmàtica o al menys una estructura de dependències. • La estructura sintagmàtica i les corresponents ‘regles’ son les que permeten que el llenguatge sigui composicional.

Però es cert això? Però es cert això?

Resum • • Historia (per entendre els objectius de les teories) Why phrase structures? Resum • • Historia (per entendre els objectius de les teories) Why phrase structures? (problemes) Why dependency grammars? (problemes) Why a probabilistic approach? (Brute force vs. theory) • Estat actual del nostre model i recerca futura.

Una previa: Com millorar els resultats? • Augmentant el training size? • Mètodes estadístics Una previa: Com millorar els resultats? • Augmentant el training size? • Mètodes estadístics més eficients? • O millorant les teories?

Historia Grammars as computational theories Historia Grammars as computational theories

Grammars as computational theories • Cognition is computation. • A grammar is a form Grammars as computational theories • Cognition is computation. • A grammar is a form of computation.

Computational theories (Marr 1980) • What is the goal of the computation? • Why Computational theories (Marr 1980) • What is the goal of the computation? • Why is it appropriate? • What is the logic of the strategy by which it can be carried out?

Chomsky’s Goal A syntactic theory has as a goal to explain the capacity of Chomsky’s Goal A syntactic theory has as a goal to explain the capacity of speakers to judge as acceptable (or ‘generate’) well formed sentences and to rule out ill-formed ones.

Justification • Syntax is indepedent of semantics. • Speakers can judge as ill or Justification • Syntax is indepedent of semantics. • Speakers can judge as ill or well-formed new sentences that they have never heard before.

Quin es el origin dels sintagmes? • No es semàntic. Un NP no es Quin es el origin dels sintagmes? • No es semàntic. Un NP no es un NP perquè es correspongui amb un argument semàntic. • Es un NP en base a trets purament sintàctics. Regularitats en la distribució de les paraules en les frases. • Tests que determinen que es un constituïen (un sintagma) i que no ho és.

Constituency Tests • “Tests of constituency are basic components of the syntactician’s toolbox. By Constituency Tests • “Tests of constituency are basic components of the syntactician’s toolbox. By investigating which strings of words can and cannot be moved, deleted, coordinated or stand in coreference relations, it is possible to draw inferences about the internal structure of sentences. ” (Phillips, 1998, p. 1)

 • Chomsky assumed that, given the independence of syntax, a theory of syntax • Chomsky assumed that, given the independence of syntax, a theory of syntax can be developed without a semantic theory and ignoring the mapping process, following only the well-formedness goal.

Mapping Goal A syntactic theory has as a goal to explain the capacity of Mapping Goal A syntactic theory has as a goal to explain the capacity of native speakers to map sentences into the corresponding conceptual representations and vice versa.

Mapping Goal • The mapping goal tries to figure out how linguistic expressions can Mapping Goal • The mapping goal tries to figure out how linguistic expressions can be mapped in the respective propositional representations in in the most simple and direct way.

Mapping Goal • (3. a) IBMP gave the company the patent. • (3. b) Mapping Goal • (3. a) IBMP gave the company the patent. • (3. b) PRED: gave; ARG 1: IBMP; ARG 2: the patent; ARG 3: the company. • (4. a) Low prices. • (4. b) PRED: low; ARG 1: prices.

Well-Formedness Goal • (3. a) IBMP gave the company the patent. • (3. b)** Well-Formedness Goal • (3. a) IBMP gave the company the patent. • (3. b)** IBMP company gave the patent. • (4. a) Low prices. • (4. b)** Prices low

Direct mapping The carpenter gave the nurse the book. PRED: gave; ARG 1: the Direct mapping The carpenter gave the nurse the book. PRED: gave; ARG 1: the carpenter; ARG 2: the book; ARG 3: the nurse.

El mapping pot ser directe en expresions simples • Aixo es cert per oracions El mapping pot ser directe en expresions simples • Aixo es cert per oracions simples. • Culicover, Peter W. and Andrzej Nowak. Dynamical Grammar. Volume Two of Foundations of Syntax. Oxford University Press. 2003. • Roger Schank i col·laboradors en els anys 70.

5941 5941 5941 5941 5941 5941 5941 5941 Mr. Nakamur cites the case of 5941 5941 5941 5941 5941 5941 5941 5941 Mr. Nakamur cites the case of a custome who wants to build a giant tourism complex in Baja and has been trying for eight years to get around Mexican restric on foreign NNP VBZ DT NN IN DT NN WP VBZ TO VB DT JJ NN NN IN NN CC VBZ VBN VBG IN CD NNS TO VB IN NNP NNS IN JJ cite want build try get - (A 0* *) (V*) (A 1* * * * * * * * * (A 0* *) (R-A 0*) (V*) (A 1* * * * *) * * * * * * * * (A 0* <-----4 NLDs *) *) *) (R-A 0*) * * * (V*) * * (A 1* * *) * * (AM-LOC** * *) * * * (V*) * * (AM-TMP** * * *) * * (A 1* * (V* * * *) * * (A 1* * * * *

Direct mapping • Per Culicover en frases mes complexes no es possible. – Mary Direct mapping • Per Culicover en frases mes complexes no es possible. – Mary loves the man who bougth the blue car – P 1: Mary loves the man. – P 2: The man bought the car. – P 3: blue car.

Direct mapping • No es possible? – Mary loves the man who bougth the Direct mapping • No es possible? – Mary loves the man who bougth the blue car – P 1: PRED: loves; ARG 1: Mary; ARG 2: the man. – P 2: PRED: bought; ARG 1: the man; ARG 2: the car. . – P 3: PRED: blue; ARG 1: car.

 • Why phrase structures? • Why dependency grammars? • Why phrase structures? • Why dependency grammars?

No son necessaries • Es pot aconseguir composicionalitat sense computar estructura sintagmàtica • Es No son necessaries • Es pot aconseguir composicionalitat sense computar estructura sintagmàtica • Es pot fer un mapping directe a la estructura predicat argument sense computar ni una estructura de dependències ni una sintagmàtica. • Es simplfica considerablement el procés de parsing i el tractament de la ambigüitat.

Temes per poder entrar a fons • • Why phrase structures? Why dependency grammars? Temes per poder entrar a fons • • Why phrase structures? Why dependency grammars? Why a probabilistic approach? (al menys la versió “brute-cutre force”)

D-Sem. Map V 1. 0 D-Sem. Map V 1. 0

Vectors and propositions • A proposition can be represented by a vector of features Vectors and propositions • A proposition can be represented by a vector of features (Hinton, 1981). • In order to represent the proposition the vector is divided into “slots”. • Each element of the proposition is represented in one slot.

Vectors and propositions Module 2 Semantic classes “Mary drives a bus” action human artifact Vectors and propositions Module 2 Semantic classes “Mary drives a bus” action human artifact entity SLOT 0 SLOT 1 SLOT 2 SLOT 3 Types & Backs Module 1 POS “Mary drives a bus” V MA N DT N SLOT 0 SLOT 1 SLOT 2 SLOT 3 Types & Backs

MODULE 1 Output Layer Yamada (2003) Nivre (2004) Magerman (1994) Ratnaparky (1999) Input Layer MODULE 1 Output Layer Yamada (2003) Nivre (2004) Magerman (1994) Ratnaparky (1999) Input Layer Input Word Slot 0 Slot 1 Slot 2 Slot 3 Type S Back, Test & Subcat.

with. Credit-card The carpenter bought a shirt with. Credit-card PUTPUT 2 3 0 1 with. Credit-card The carpenter bought a shirt with. Credit-card PUTPUT 2 3 0 1 PUTPUT 2 3 1 PUT Output Hidden DT MA C V DT PE IIN C N N PA Slot 1 Slot 0 Slot 2 V MA PE PA DT N C Subcategorization backtracking Slot 3 IIN N C

Module 2 supervises argument position MODULE 1 P 1) PRED: ARG 1: ARG 2: Module 2 supervises argument position MODULE 1 P 1) PRED: ARG 1: ARG 2: ARG 3: V MA PE PA ( bought) N PR (Mary) DT N C ( a shirt ) IIN N C (with pockets) MODULE 2 P 1) PRED: ARG 1: ARG 2: ARG 3: get, transfer, give, pay, entity, person entity, object, artifact (shirt) artifact, part-of-dress Subcategorization and Selectional Restrictions Parsing strategy: Attaches first to the current proposition

Binding problem Binding problem

“Mary bought a shirt with pockets” MODULE 1 P 1) PRED: ARG 1: ARG “Mary bought a shirt with pockets” MODULE 1 P 1) PRED: ARG 1: ARG 2: ARG 3: MODULE 2 V MA PE PA ( bought) N PR (Mary) DT N C ( a shirt ) IIN N C (with pockets) P 1) PRED: ARG 1: ARG 2: ARG 3: get, transfer, give, pay, entity, person entity, object, artifact part-of-dress Parsing strategy: Attaches first to the current proposition CLEARP IZ_IN 0 BACK

“Mary bought a shirt with pockets” MODULE 1 P 1: PRED: ARG 1: ARG “Mary bought a shirt with pockets” MODULE 1 P 1: PRED: ARG 1: ARG 2: ARG 3: V MA PE ( bought) N PR (Mary) DT N C ( a shirt ) P 2: PRED: ARG 1: DT N C (a shirt) ARG 2: IIN N C (with pockets) ARG 3: MODULE 2 P 1: PRED: get, transfer, pay, accept. . . ARG 1: entity, person, . . . ARG 2: entity, object, artifact, shirt ARG 3: P 2: PRED: ARG 1: entity, object, artifact, shirt ARG 2: artifact part-of-dress ARG 3:

PARSING COMPLEX SENTENCES PARSING COMPLEX SENTENCES

Elementary expressions Elementary expressions

“a blue shirt” MODULE 1 MODULE 2 PRED: JJ ( blue ) PRED: colour, “a blue shirt” MODULE 1 MODULE 2 PRED: JJ ( blue ) PRED: colour, blue (SLOT 0) ARG 1: DT N C ( a shirt ) ARG 1: entity, object, artifact (SLOT 1) “the governement`s minister” ARG 1: DT N C ( the minister ) (SLOT 1) ARG 2: N C POS ( gover. DTent’s ) (SLOT 2) TYPE: POS (SLOT type) ARG 1: entity, person. . . (SLOT 1) ARG 2: entity, person. . . (SLOT 2) TYPE: POS (SLOT type)

Complex sentences • A complex sentence is any sentence that is formed by more Complex sentences • A complex sentence is any sentence that is formed by more than one elementary expression • A complex sentence requires more than one proposition for its semantic representation

MODULE 1 Output Layer Input Word Slot 0 Slot 1 Slot 2 Slot 3 MODULE 1 Output Layer Input Word Slot 0 Slot 1 Slot 2 Slot 3 Type S Back, Test & Subcat.

Non Invariant Solution Output Layer Input Word COMPLETE SENTENCE STRUCTURE (OPERATIONS WITH VECTORS) Non Invariant Solution Output Layer Input Word COMPLETE SENTENCE STRUCTURE (OPERATIONS WITH VECTORS)

Invariant solution (a kind of shift and reduce parser) Output Layer STACK Stored Context Invariant solution (a kind of shift and reduce parser) Output Layer STACK Stored Context Input Layer Input Word Slot 0 Slot 1 Slot 2 Slot 3 Type S Back, Test & Subcat. Focus of attention (current context)

MODULE 1 Output Layer STACK Stored Context Input Layer Input Word Slot 0 Slot MODULE 1 Output Layer STACK Stored Context Input Layer Input Word Slot 0 Slot 1 Slot 2 Slot 3 Type S Back, Test & Subcat. Focus of attention (current context)

Modelos concéntricos (Cowan, 1988, 1995, 1999; Oberauer, 2002) parte activada de la MLP foco Modelos concéntricos (Cowan, 1988, 1995, 1999; Oberauer, 2002) parte activada de la MLP foco de atención memoria a largo plazo (MLP)

A Neurons whose receptive fields are invariant (translation and scale), higher visual areas (inferotemporal A Neurons whose receptive fields are invariant (translation and scale), higher visual areas (inferotemporal cortex) Covert attention retinotopic visual neurons (as found in V 1 and V 2) A A J L

A Neurons whose receptive fields are invariant (translation and scale), higher visual areas (inferotemporal A Neurons whose receptive fields are invariant (translation and scale), higher visual areas (inferotemporal cortex) Covert attention A retinotopic visual neurons (as found in V 1 and V 2) A J L

Attention and invariance • Shift reduce parser • Stolke (1990), Sopena (1993), Miikulainen (1996) Attention and invariance • Shift reduce parser • Stolke (1990), Sopena (1993), Miikulainen (1996)

The main manager bought some old cars with three wheels. Input Word: The| DT The main manager bought some old cars with three wheels. Input Word: The| DT OUTPUT M 1: PUT 1 M 2: Current Pred: A 1: A 2: A 3: Flags:

Generalized Role Labeling using Propositional Representations The main manager bought some old cars with Generalized Role Labeling using Propositional Representations The main manager bought some old cars with three wheels. Input Word: The| DT M 1: NEXT M 2: Current Pred: A 1: The A 2: A 3: Flags: @1

Generalized Role Labeling using Propositional Representations The main manager bought some old cars with Generalized Role Labeling using Propositional Representations The main manager bought some old cars with three wheels. Input Word: main | JJ_PR M 1: *IZ-IN M 2: Current Pred: A 1: The A 2: A 3: Flags: @NEXT @1

Generalized Role Labeling using Propositional Representations The main manager bought some old cars with Generalized Role Labeling using Propositional Representations The main manager bought some old cars with three wheels. Input Word: main| JJ_PR M 1: PUT 0 M 2: Current Top Pred: A 1: The A 2: A 3: Flags: @1

Generalized Role Labeling using Propositional Representations The main manager bought some old cars with Generalized Role Labeling using Propositional Representations The main manager bought some old cars with three wheels. Input Word: main| JJ_PR M 1: NEXT M 2: Current Top Pred: main Pred: A 1: The A 2: A 3: Flags: @1

Generalized Role Labeling using Propositional Representations The main manager bought some old cars with Generalized Role Labeling using Propositional Representations The main manager bought some old cars with three wheels. Input Word: manager| DT_N M 1: PUT 1 M 2: Current Top Pred: main Pred: A 1: The A 2: A 3: Flags: @NEXT Flags: @1

Generalized Role Labeling using Propositional Representations The main manager bought some old cars with Generalized Role Labeling using Propositional Representations The main manager bought some old cars with three wheels. Input Word: manager| DT_N M 1: OZ-OUT M 2: Current Top Pred: main Pred: A 1: manager A 1: The A 2: A 3: Flags: @NEXT Flags: @1

Generalized Role Labeling using Propositional Representations The main manager bought some old cars with Generalized Role Labeling using Propositional Representations The main manager bought some old cars with three wheels. Input Word: manager| DT_N M 1: PUT 1 M 2: Current Pred: A 1: The A 2: A 3: Flags: @1 @OZ-OUT P: main|A 1: manager

Generalized Role Labeling using Propositional Representations The main manager bought some old cars with Generalized Role Labeling using Propositional Representations The main manager bought some old cars with three wheels. Input Word: manager| DT_N M 1: NEXT M 2: Current Pred: A 1: The manager A 2: A 3: Flags: @1 @OZ-OUT P: main|A 1: manager

Generalized Role Labeling using Propositional Representations The main manager bought some old cars with Generalized Role Labeling using Propositional Representations The main manager bought some old cars with three wheels. Input Word: bought| V_MA M 1: PUT 0 M 2: Current Pred: A 1: The manager A 2: A 3: Flags: @1 @NEXT P: main|A 1: manager

Generalized Role Labeling using Propositional Representations The main manager bought some old cars with Generalized Role Labeling using Propositional Representations The main manager bought some old cars with three wheels. Input Word: bought| V_MA M 1: NEXT M 2: Current Pred: bought A 1: The manager A 2: A 3: Flags: @0 P: main|A 1: manager

Generalized Role Labeling using Propositional Representations The main manager bought some old cars with Generalized Role Labeling using Propositional Representations The main manager bought some old cars with three wheels. Input Word: some| DT M 1: PUT 2 M 2: Current Pred: bought A 1: The manager A 2: A 3: Flags: @0 @NEXT P: main|A 1: manager

Generalized Role Labeling using Propositional Representations The main manager bought some old cars with Generalized Role Labeling using Propositional Representations The main manager bought some old cars with three wheels. Input Word: some| DT M 1: NEXT M 2: Current Pred: bought A 1: The manager A 2: some A 3: Flags: @2 P: main|A 1: manager

Generalized Role Labeling using Propositional Representations The main manager bought some old cars with Generalized Role Labeling using Propositional Representations The main manager bought some old cars with three wheels. Input Word: old| JJ_PR M 1: *IZ-IN M 2: Current Pred: bought A 1: The manager A 2: some A 3: Flags: @2 @NEXT P: main|A 1: manager

Generalized Role Labeling using Propositional Representations The main manager bought some old cars with Generalized Role Labeling using Propositional Representations The main manager bought some old cars with three wheels. Input Word: old| JJ_PR M 1: PUT 0 M 2: Current Top Pred: bought A 1: The manager A 2: some A 3: Flags: @IZ-IN Flags: @2 P: main|A 1: manager

Compositionality • “We now turn to what I think was an important mistake at Compositionality • “We now turn to what I think was an important mistake at the core of generative grammar, one that in retrospect lies behind much of the alienation of linguistic theory from the cognitive sciences. Chomsky did demonstrate that language requires a generative system that makes possible an infinite variety of sentences. However, he explicitly assumed, without argument (1965: 16, 17, 75, 198), that generativity is localized in the syntactic component of the grammar” (Jackendoff, 2002)

Compositionality • The fact that semantics is ‘purely interpretive’ has as a consequence that Compositionality • The fact that semantics is ‘purely interpretive’ has as a consequence that thought has no ‘independent status’ and it cannot be creative or have an independent capacity of combinatoriality outside of language. • As Phillips (2004) points out, that thought is purely interpretative and not creative is “a consequence that is likely to be uncomfortable for many, including Jackendoff” (Phillips, 2004 p. 574).

Compositionality • Semantic (or thought) is “purely interpretative”. Only syntax is creative. • I Compositionality • Semantic (or thought) is “purely interpretative”. Only syntax is creative. • I think that to believe in ‘absolute free will’ is the main cause of all types of fundamentalism,

Training minimalista • • • SS-1 -1 - (DT The WAIT NEXT) SS-1 -2 Training minimalista • • • SS-1 -1 - (DT The WAIT NEXT) SS-1 -2 - (DT_N man PUT 1 NEXT) SS-1 -3 - (V_MA sold PUT 0 NEXT) SS-1 -4 - (DT some WAIT NEXT) SS-1 -5 - (DT_N offerings PUT 2 NEXT) SS-1 -6 - (IIN_DT to WAIT NEXT) SS-1 -7 - (DT the WAIT NEXT) SS-1 -8 - (DT_N president PUT 3 NEXT) SS-1 -9 - (. . OZ-OUT NEXT) SS-1 -10 - (FIN)

Training minimalista (8 -10 paraules maxim) • • • RL-22 -1 - (DT a Training minimalista (8 -10 paraules maxim) • • • RL-22 -1 - (DT a NADA NEXT) RL-22 -2 - (DT_N land PUT 1 NEXT) RL-22 -3 - (CC , PUTtype. CC NEXT) RL-22 -4 - (WP 2 where TESTARG &NOTEST CLEARmode. CC IZ-IN 2 PUTtype. WDT NEXT) RL-22 -5 - (DT a NADA NEXT) RL-22 -6 - (DT_N saying PUT 1 NEXT) RL-22 -7 - (V_MA says PUT 0 &BACK 2 MV 23 &BACK_ADJ IZ-INE 2 PUTtype. ADJ OZ-OUT NEXT) RL-22 -8 - (. . OZ-OUT NEXT) RL-22 -9 - (FIN)

Test real, PTBII (55 -26) • s 5974: But predictions that central banks of Test real, PTBII (55 -26) • s 5974: But predictions that central banks of the Group of Seven - G-7 - major industrial nations would continue their massive dollar sales went astray , as the market drove the dollar downward on its own , reacting to Wall Street 's plunge and subsequent price volatility , lower U. S. interest rates and signs of a slowing U. S. economy.

Test • Ho hem probat amb 254 frases del PTBII • Els resultats son Test • Ho hem probat amb 254 frases del PTBII • Els resultats son molt bons. La idea es tenir 0% errors i crec que es pot conseguir. • Quin es le problema?

NLDs, coordination, comparatives, puntuació • Dependency grammars and parsers often ignore some classes of NLDs, coordination, comparatives, puntuació • Dependency grammars and parsers often ignore some classes of dependencies • Puntuació (guionets, parentesis, comes, dos punts, . . . )

NLDs NLDs

5941 5941 5941 5941 5941 5941 5941 5941 Mr. Nakamur cites the case of 5941 5941 5941 5941 5941 5941 5941 5941 Mr. Nakamur cites the case of a custome who wants to build a giant tourism complex in Baja and has been trying for eight years to get around Mexican restric on foreign NNP VBZ DT NN IN DT NN WP VBZ TO VB DT JJ NN NN IN NN CC VBZ VBN VBG IN CD NNS TO VB IN NNP NNS IN JJ cite want build try get - (A 0* *) (V*) (A 1* * * * * * * * * (A 0* *) (R-A 0*) (V*) (A 1* * * * *) * * * * * * * * (A 0* <-----4 NLDs *) *) *) (R-A 0*) * * * (V*) * * (A 1* * *) * * (AM-LOC** * *) * * * (V*) * * (AM-TMP** * * *) * * (A 1* * (V* * * *) * * (A 1* * * * *

5961 5961 5961 5961 5961 5961 5961 Fed funds is the rate banks charge 5961 5961 5961 5961 5961 5961 5961 Fed funds is the rate banks charge each other on overnig loans ; the Fed influen the rate by adding or drainin reserve from the banking system. NNP NNS VBZ DT NN NNS VBP DT JJ IN JJ NNS : DT NNP VBZ DT NN IN VBG CC VBG NNS IN DT NN NN. * * * (A 1* *) (A 0*) charge (V*) (A 2* *) (A 3* * *) * * * influenc* * add * * drain * * * * * * * * * * * * (A 0* *) *) *) (V*) * * (A 1* * * *) * * (AM-MNR** * * (V*) * (A 1*) (A 2*) * (AM-MNR*(A 1* * * * *) *) *) * * *

5920 5920 5920 5920 5920 5920 5920 5920 For the PRI to stand a 5920 5920 5920 5920 5920 5920 5920 5920 For the PRI to stand a chance , Mr. Salinas has to press on with an economi program that so far has succeed in lowerin inflati and providi moderat economi growth. IN DT NNP TO VB DT NN , NNP VBZ TO VB RP IN DT JJ NN WDT RB RB VBZ VBN IN VBG NN CC VBG JJ JJ NN. stand have press succeed lower provide - * (A 0* *) * (V*) (A 1* *) * * * * * * * * * (V*) * * * * * * (AM-PNC * * *) * (A 0* *) (AM-MOD * (V* *) (A 1* * * * *) * * * * * * * * (A 0* * * *) *) (R-A 0*) (AM-TMP** *) * * * (V*) * (A 1*) * * * * * * * (A 0* * *) (R-A 0*) <----- 3 NLDs * * * * (V*) (A 1* * *) *

NLDs (Johnson 2002) • Broad coverage syntactic parsers with good performance have recently become NLDs (Johnson 2002) • Broad coverage syntactic parsers with good performance have recently become available (Charniak, Collins), but these typically produce as output a parse tree that only encodes local syntactic information, i. e. , a tree that does not include any "empty nodes".

NLDs (Dienes 2003) • Intuitively, the problem of parsing with NLDs is that the NLDs (Dienes 2003) • Intuitively, the problem of parsing with NLDs is that the empty elements (EEs) representing these dependencies are not in the input. • Therefore, the parser has to hypothesise where these EEs might occur – in the worst case, it might end up suggesting exponentially many traces, rendering parsing infeasible (Johnson and Kay 1994).

NLDs • From the point of view of the dependency structure, NLDs are difficult NLDs • From the point of view of the dependency structure, NLDs are difficult because they violate the assumption that dependency structures are represented as directed trees. Specifically, NLDs give rise to re-entrancies in the dependency graph, i. e. , it is no longer a directed tree but a directed graph, with nodes possibly having multi- ple parents (e. g. apple in Figure 2. 2). Now, the parser has to explore a much larger search space.

NLDs • Arguably, the search space is much more restricted by an actual grammar NLDs • Arguably, the search space is much more restricted by an actual grammar that exploits, for instance, the knowledge that buy is a transitive verb and thus requires a direct object. • Nevertheless, the problem does not disappear. Consider the following example: When demand is stronger than suppliers can handle and delivery times lengthen, prices tend to rise. (wsj_0036. mrg)

NLDs • Non-local dependencies and displacement phenomena have been a central topic of generative NLDs • Non-local dependencies and displacement phenomena have been a central topic of generative linguistics since its inception half a century ago. However … Many current linguistic theories of nonlocal dependencies are extremely complex, and would be difficult to apply with the kind of broad coverage described here.

Why a probabilistic approach? • “Ambiguity and underspecification are ubiquitous in human language utterances, Why a probabilistic approach? • “Ambiguity and underspecification are ubiquitous in human language utterances, at all levels (lexical, syntactic, semantic, etc. ), and how to resolve these ambiguities is a key communicative task for both human and computer natural language understanding” (Manning, 2003).

 • At the highest level, the probabilistic approach to natural language understanding is • At the highest level, the probabilistic approach to natural language understanding is to view the task as trying to learn the probability distribution: – P(meaning| utterance; context) • A mapping from form to meaning conditioned by context.

Why a probabilistic approach quantum mechanics (uncertainty) Classical Physics (underspecification) Why a probabilistic approach quantum mechanics (uncertainty) Classical Physics (underspecification)

Why a probabilistic approach? • Collins (1996) – Ambiguity: • PP-attachment • Coordination Why a probabilistic approach? • Collins (1996) – Ambiguity: • PP-attachment • Coordination

PP-attachment • Pierre Vinken, 61 years old, joined the board as a nonexecutive director. PP-attachment • Pierre Vinken, 61 years old, joined the board as a nonexecutive director.

PP-attachment • 4 -tuple joined board as director • V = joined, N 1 PP-attachment • 4 -tuple joined board as director • V = joined, N 1 = board, P = as, and N 2 = director. • p (A= l | V=v, N 1=n 1, P=p, N 2=n 2) • p(l | v, n 1, p, n 2)

Results PP attachment ordered by Accuracy Method Accuracy Ratnaparkhi (1994) ID 3 Zavrel et Results PP attachment ordered by Accuracy Method Accuracy Ratnaparkhi (1994) ID 3 Zavrel et al. (1996) Neural Networks Ratnaparkhi (1994) Maximum Entropy Model Takahashi (2001) Neural Networks Yeh and Vilain (1998) Error-driven learning Abney et al. (1999) Boosting Zavrel et al. (1997) Memory-Based Learning Collins and Brooks (1995) Backed-Off Model Krymolowski and Rooth (1998) SNOW Committee Machines 1 (Alegre et al, 1999) 77. 70 % 80. 00 % 81. 60 % 83. 10 % 84. 40 % 84. 50 % 84. 80 % 86. 08 % Committee Machines 2 (Alegre, 2004) 88. 01 % Average Human Expert (Ratnaparkhi, 1994) 88. 20 %

“Mary bought a shirt with pockets” MODULE 1 P 1) PRED: ARG 1: ARG “Mary bought a shirt with pockets” MODULE 1 P 1) PRED: ARG 1: ARG 2: ARG 3: MODULE 2 V MA PE PA ( bought) N PR (Mary) DT N C ( a shirt ) IIN N C (with pockets) P 1) PRED: ARG 1: ARG 2: ARG 3: get, transfer, give, pay, entity, person entity, object, artifact part-of-dress Parsing strategy: Attaches first to the current proposition CLEARP IZ_IN 0 BACK

“Mary bought a shirt with pockets” MODULE 1 P 1: PRED: ARG 1: ARG “Mary bought a shirt with pockets” MODULE 1 P 1: PRED: ARG 1: ARG 2: ARG 3: V MA PE ( bought) N PR (Mary) DT N C ( a shirt ) P 2: PRED: ARG 1: DT N C (a shirt) ARG 2: IIN N C (with pockets) ARG 3: MODULE 2 P 1: PRED: get, transfer, pay, accept. . . ARG 1: entity, person, . . . ARG 2: entity, object, artifact, shirt ARG 3: P 2: PRED: ARG 1: entity, object, artifact, shirt ARG 2: artifact part-of-dress ARG 3:

Millors resultats • Alegre (2002) – PTBI 89. 8% (PTBII 92. 3%) • Olteanu Millors resultats • Alegre (2002) – PTBI 89. 8% (PTBII 92. 3%) • Olteanu and Modovan (2005) – PTBII 94%

Parsers de dependencies? Parsers de dependencies?

Ambigüitat • Tota la “artilleria” que cal per resoldre el PP -attachment s’hauria de Ambigüitat • Tota la “artilleria” que cal per resoldre el PP -attachment s’hauria de fer servir per resoldre com col. locar cada paraula de la frase en la estructura? • En quins fenòmens caldria? • La resposta es en molt pocs.