Скачать презентацию PSY 369 Psycholinguistics Language Comprehension Sentence comprehension Скачать презентацию PSY 369 Psycholinguistics Language Comprehension Sentence comprehension

97f79252bb561d094da557025f8e7278.ppt

  • Количество слайдов: 88

PSY 369: Psycholinguistics Language Comprehension: Sentence comprehension PSY 369: Psycholinguistics Language Comprehension: Sentence comprehension

Overview of comprehension Input The cat chased the rat. Language perception c a t Overview of comprehension Input The cat chased the rat. Language perception c a t /k/ /ae/ /t/ Word recognition cat dog cap wolf tree yarn cat claw fur hat Syntactic Semantic & pragmatic analysis S NP VP the cat V NP chased the rat

Comprehending sentences The man hit the dog with the leash. S S NP NP Comprehending sentences The man hit the dog with the leash. S S NP NP VP V NP N The man hit V PP Instrument NP det N with the leash the dog n NP PP NP det The N man hit det Modifier N the dog with Theory question n n VP How do we know which structure to build? Methodological question n What methods can we use to help us answer theoretical questions like these? the leash

Commonly used methods n Remember, the “output” of comprehension processes are internal mental events. Commonly used methods n Remember, the “output” of comprehension processes are internal mental events. What kinds of measures have been used to test theories? n Cross-modal priming n n Comprehension measure n n Read a sentence/passage and then answer questions about what you read (decision time, accuracy) Measure how long people actually spend reading n n E. g. , listen to a sentence, make a lexical decision to a visually presented word Line by line reading Word by word reading Using eye movement monitoring techniques Neuropsychological methods n E. g. , ERPs, f. MRI

Line-by-line A banker is a fellow Line-by-line A banker is a fellow

Line-by-line who lends you his umbrella Line-by-line who lends you his umbrella

Line-by-line when the sun is shining Line-by-line when the sun is shining

Line-by-line but wants it back Line-by-line but wants it back

Line-by-line the minute it begins to rain. Line-by-line the minute it begins to rain.

Line-by-line n Problem: n n Overall reading time for entire sentence or phrase need Line-by-line n Problem: n n Overall reading time for entire sentence or phrase need for more “on-line” measurements n Timing on a smaller scope n See effects at level of word

Word-by-word n A couple of methods n n RSVP (rapid serial visual presentation) Moving Word-by-word n A couple of methods n n RSVP (rapid serial visual presentation) Moving window

Word-by-word n RSVP (rapid serial visual presentation) n n Experimenter presents the sentence one Word-by-word n RSVP (rapid serial visual presentation) n n Experimenter presents the sentence one word a time. Typically the experimenter controls the presentation rate.

Word-by-word A Word-by-word A

Word-by-word lie Word-by-word lie

Word-by-word can Word-by-word can

Word-by-word travel Word-by-word travel

Word-by-word halfway Word-by-word halfway

Word-by-word around Word-by-word around

Word-by-word the Word-by-word the

Word-by-word world Word-by-word world

Word-by-word while Word-by-word while

Word-by-word the Word-by-word the

Word-by-word truth Word-by-word truth

Word-by-word is Word-by-word is

Word-by-word putting Word-by-word putting

Word-by-word on Word-by-word on

Word-by-word its Word-by-word its

Word-by-word shoes. Word-by-word shoes.

Word-by-word n Moving window Word-by-word n Moving window

Word-by-word I xxxxx xx xxxxxxxxx xx xxxxx. Word-by-word I xxxxx xx xxxxxxxxx xx xxxxx.

Word-by-word x have xxxx xx xxxxxxxxx xx xxxxx. Word-by-word x have xxxx xx xxxxxxxxx xx xxxxx.

Word-by-word x xxxx never xxx xx xxxxxxxxx xx xxxxx. Word-by-word x xxxx never xxx xx xxxxxxxxx xx xxxxx.

Word-by-word x xxxxx let xx xxxxxxxxx xx xxxxx. Word-by-word x xxxxx let xx xxxxxxxxx xx xxxxx.

Word-by-word x xxxxx xxx my xxxxxxxxx xx xxxxx. Word-by-word x xxxxx xxx my xxxxxxxxx xx xxxxx.

Word-by-word x xxxxx xx schooling xxxxx xx xxxxx. Word-by-word x xxxxx xx schooling xxxxx xx xxxxx.

Word-by-word x xxxxx xx xxxxx interfere xxxx xx xxxxx. Word-by-word x xxxxx xx xxxxx interfere xxxx xx xxxxx.

Word-by-word x xxxxx xx xxxxxxxxx with xx xxxxx. Word-by-word x xxxxx xx xxxxxxxxx with xx xxxxx.

Word-by-word x xxxxx xx xxxxxxxxx my xxxxx. Word-by-word x xxxxx xx xxxxxxxxx my xxxxx.

Word-by-word x xxxxx xx xxxxxxxxx xx education. Word-by-word x xxxxx xx xxxxxxxxx xx education.

Word-by-word n A couple of methods n n RSVP (rapid serial visual presentation) Moving Word-by-word n A couple of methods n n RSVP (rapid serial visual presentation) Moving window Better, more “on-line” But, these measures are also a little bit unnatural (especially RSVP) n e. g. , Don’t allow regressions (looking back)

Eye-movements in reading n One of the most common measures used in sentence comprehension Eye-movements in reading n One of the most common measures used in sentence comprehension research is measuring Eye-movements

How the eye “works” n n At its center is the fovea, a pit How the eye “works” n n At its center is the fovea, a pit that is most sensitive to light and is responsible for our sharp central vision. The central retina is conedominated and the peripheral retina is rod-dominated.

How the eye “works” n Limitations of the visual field n 130 degrees vertically, How the eye “works” n Limitations of the visual field n 130 degrees vertically, 180 degrees horizontally (including peripheral vision n Perceptual span for reading: 7 -12 spaces Clothes make the man. Naked people have little or no influence on society.

How the eye “works” n Within the visual field, eye movements serve two major How the eye “works” n Within the visual field, eye movements serve two major functions Saccades to Fixations – Position target objects of interest on the fovea Tracking – Keep fixated objects on the fovea despite movements of the object or head n n n Eye-movements in reading are saccadic rather than smooth Clothes make the man. Naked people have little or no influence on society. Video examples: 1|2|3|4|5

Smooth Pursuit n n Smooth movement of the eyes for visually tracking a moving Smooth Pursuit n n Smooth movement of the eyes for visually tracking a moving object Cannot be performed in static scenes (fixation/saccade behavior instead)

Saccades n Saccades are used to move the fovea to the next object/region of Saccades n Saccades are used to move the fovea to the next object/region of interest. n n Connect fixations Duration 10 ms - 120 ms n n Very fast (up to 700 degrees/second) Ballistic movements (pre-programmed) About 150, 000 saccades per day No visual perception during saccades n n Vision is suppressed Evidence that some cognitive processing may also be suppressed during eye-movements (Irwin, 1998)

Saccades Without suppression Move to here With suppression Move to here Video example (around Saccades Without suppression Move to here With suppression Move to here Video example (around 5 min mark)

Smooth Pursuit versus Saccades n n n n Jerky No correction Up to 700 Smooth Pursuit versus Saccades n n n n Jerky No correction Up to 700 degrees/sec Background is not blurred (saccadic suppression) n Smooth pursuit n n Smooth and continuous Constantly corrected by visual feedback Up to 100 degrees/sec Background is blurred

Fixations n The eye is (almost) still – perceptions are gathered during fixations n Fixations n The eye is (almost) still – perceptions are gathered during fixations n n n 90% of the time the eye is fixated duration: 150 ms - 600 ms In reading, the assumption is that the length of fixation is correlated with amount/type of processing being done at that point (on that word, at that point in the syntactic parse) Video examples: 1|2|3|4|5 n Article/video (if I can get it to work) n Raney, G. E. , Campbell, S. J. , Bovee, J. C. (2014). Using Eye Movements to Evaluate the Cognitive Processes Involved in Text Comprehension. J. Vis. Exp. (83), e 50780, doi: 10. 3791/50780.

Measuring Eye Movements Purkinje Eye Tracker n n n Laser is aimed at the Measuring Eye Movements Purkinje Eye Tracker n n n Laser is aimed at the eye. Laser light is reflected by cornea and lens Pattern of reflected light is received by an array of lightsensitive elements. Very precise Also measures pupil accommodation (switching between looking far or close) No head movements

Measuring Eye Movements Video-Based Systems n n Infrared camera directed at eye Image processing Measuring Eye Movements Video-Based Systems n n Infrared camera directed at eye Image processing hardware determines pupil position and size (and possibly corneal reflection) Good spatial precision (0. 5 degrees) for head-mounted systems Good temporal resolution (up to 500 Hz) possible

Eye movements in reading The man hit the dog with the leash. S S Eye movements in reading The man hit the dog with the leash. S S NP NP VP V NP N The man hit V PP Instrument NP det N with the leash the dog n NP PP NP det The N man hit det Modifier N the dog with Theory question n n VP How do we know which structure to build? Methodological question n How can we use eye-movements to help us answer theoretical question the leash

Parsing n The syntactic analyser or “parser” n n Main task: To construct a Parsing n The syntactic analyser or “parser” n n Main task: To construct a syntactic structure from the words of the sentence as they arrive Main research question: how does the parser “make decisions” about what structure to build?

Eye movements in reading The man hit the dog with the leash. S NP Eye movements in reading The man hit the dog with the leash. S NP det N The man

Eye movements in reading The man hit the dog with the leash. S NP Eye movements in reading The man hit the dog with the leash. S NP VP V det N The man hit

Eye movements in reading The man hit the dog with the leash. S NP Eye movements in reading The man hit the dog with the leash. S NP VP V NP NP det N The man hit the dog

Eye movements in reading The man hit the dog with the leash. S NP Eye movements in reading The man hit the dog with the leash. S NP VP V NP NP det N The man hit the dog PP Modifier with the leash

Eye movements in reading The man hit the dog with the leash. S NP Eye movements in reading The man hit the dog with the leash. S NP VP V NP NP det N The man hit the dog PP Instrument with the leash

Sentence Comprehension n n There are many examples of syntactic ambiguity A vast amount Sentence Comprehension n n There are many examples of syntactic ambiguity A vast amount of research focuses on: Garden path sentences n A garden path sentence invites the listener to consider one possible parse, and then at the end forces him to abandon this parse in favor of another. “The horse raced past the barn fell. ” Actual Newspaper Headlines Veterinarian Takes n Panda Mating Fails; n n n Juvenile Court to Try Shooting Defendant Red tape holds up new bridge Miners Refuse to Work after Death Retired priest may marry Springsteen Local High School Dropouts Cut in Half n n n Over Kids Make Nutritious Snacks Squad Helps Dog Bite Victim Hospitals are Sued by 7 Foot Doctors

Different approaches n Immediacy Principle: access the meaning/syntax of the word and fit it Different approaches n Immediacy Principle: access the meaning/syntax of the word and fit it into a syntactic structure n Serial Analysis (Modular): Build just one based on syntactic information and continue to try to add to it as long as this is still possible

Sentence Comprehension n The horse raced past the barn fell. S NP The horse Sentence Comprehension n The horse raced past the barn fell. S NP The horse VP

Sentence Comprehension n The horse raced past the barn fell. S NP VP V Sentence Comprehension n The horse raced past the barn fell. S NP VP V The horse raced

Sentence Comprehension n The horse raced past the barn fell. S NP VP V Sentence Comprehension n The horse raced past the barn fell. S NP VP V PP P The horse raced past NP

Sentence Comprehension n The horse raced past the barn fell. S NP VP V Sentence Comprehension n The horse raced past the barn fell. S NP VP V PP P NP The horse raced past the barn

Sentence Comprehension n The horse raced past the barn fell. S NP VP V Sentence Comprehension n The horse raced past the barn fell. S NP VP V PP P NP The horse raced past the barn fell

Sentence Comprehension n The horse raced past the barn fell. n S NP VP Sentence Comprehension n The horse raced past the barn fell. n S NP VP V PP P NP The horse raced past the barn raced is initially treated as a past tense verb

Sentence Comprehension n The horse raced past the barn fell. n n S NP Sentence Comprehension n The horse raced past the barn fell. n n S NP VP V PP P NP The horse raced past the barn fell raced is initially treated as a past tense verb This analysis fails when the verb fell is encountered

Sentence Comprehension n The horse raced past the barn fell. n n S NP Sentence Comprehension n The horse raced past the barn fell. n n S NP n raced is initially treated as a past tense verb This analysis fails when the verb fell is encountered raced can be re-analyzed as a past participle. VP V S PP P VP NP NP The horse raced past the barn fell NP V RR PP V P NP The horse raced past the barn fell

Different approaches n Immediacy Principle: access the meaning/syntax of the word and fit it Different approaches n Immediacy Principle: access the meaning/syntax of the word and fit it into a syntactic structure n n Serial Analysis (Modular): Build just one based on syntactic information and continue to try to add to it as long as this is still possible Interactive Analysis: Use multiple levels (both syntax and semantics) of information to build the “best” structure

Different approaches n Immediacy Principle: access the meaning/syntax of the word and fit it Different approaches n Immediacy Principle: access the meaning/syntax of the word and fit it into a syntactic structure n n Serial Analysis (Modular): Build just one based on syntactic information and continue to try to add to it as long as this is still possible Interactive Analysis: Use multiple levels (both syntax and semantics) of information to build the “best” structure Parallel Analysis: Build both alternative structures at the same time Minimal Commitment: Stop building - and wait until later material clarifies which analysis is the correct one.

A serial model n Formulated by Lyn Frazier (1978, 1987) n Build trees using A serial model n Formulated by Lyn Frazier (1978, 1987) n Build trees using syntactic cues: n n phrase structure rules plus two parsing principles n n n Minimal Attachment Late Closure Go back and revise the syntax if later semantic information suggests things were wrong

A serial model n Minimal Attachment n Prefer the interpretation that is accompanied by A serial model n Minimal Attachment n Prefer the interpretation that is accompanied by the simplest structure. n simplest = fewest branchings (tree metaphor!) n Count the number of nodes = branching points The girl hit the man with the umbrella.

Minimal attachment S 8 Nodes NP Preferred the girl S NP the girl NP Minimal attachment S 8 Nodes NP Preferred the girl S NP the girl NP hit the man V hit VP V VP NP NP the man PP P NP with the umbrella The girl hit the man with the umbrella. 9 nodes

A serial model n Late Closure n Incorporate incoming material into the phrase or A serial model n Late Closure n Incorporate incoming material into the phrase or clause currently being processed. OR n Associate incoming material with the most recent material possible. She said he tickled her yesterday

Parsing Preferences. . late closure S Preferred S np np vp she v S' Parsing Preferences. . late closure S Preferred S np np vp she v S' said np he vp said np vp v np S' he adv tickled her yesterday She said he tickled her yesterday adv vp v yesterday np tickled her (Both have 10 nodes, so use LC not MA)

Minimal attachment n Garden path sentences (Rayner & Frazier, ‘ 83) The spy saw Minimal attachment n Garden path sentences (Rayner & Frazier, ‘ 83) The spy saw the cop with a telescope. minimal attach non-minimal attach Modular prediction Build this structure first Interactive prediction Build this structure first

Minimal attachment n Garden path sentences (Rayner & Frazier, ‘ 83) The spy saw Minimal attachment n Garden path sentences (Rayner & Frazier, ‘ 83) The spy saw the cop with a revolver. minimal attach non-minimal attach Modular prediction Build this structure first Interactive prediction Build this structure first Lexical/semantic information rules this one out

MA S S NP the spy NP VP S’ the spy V PP NP MA S S NP the spy NP VP S’ the spy V PP NP saw P Non-MA VP V NP saw NP PP the cop P NP the cop with the revolver but the cop didn’t see him S’ NP with the revolver but the cop didn’t see him The spy saw the cop with the binoculars. . The spy saw the cop with the revolver … <- takes longer to read (Rayner & Frazier, ‘ 83)

Interactive Models n n Other factors (e. g. , semantic context, co-occurrence of usage Interactive Models n n Other factors (e. g. , semantic context, co-occurrence of usage & expectation) may provide cues about the likely interpretation of a sentence Trueswell et al (1994) n n The evidence examined by the lawyer … The defendant examined by the lawyer… evidence typically gets examined, but can’t do the examining

Interactive Models n n Other factors (e. g. , semantic context, co-occurrence of usage Interactive Models n n Other factors (e. g. , semantic context, co-occurrence of usage & expectation) may provide cues about the likely interpretation of a sentence Trueswell et al (1994) n n The evidence examined by the lawyer … The defendant examined by the lawyer … A defendant can be examined but can also do examining.

Semantic expectations n n Other factors (e. g. , semantic context, co-occurrence of usage Semantic expectations n n Other factors (e. g. , semantic context, co-occurrence of usage & expectation) may provide cues about the likely interpretation of a sentence Taraban & Mc. Celland (1988) n n n Expectation The couple admired the house with a friend but knew that it was over-priced. The couple admired the house with a garden but knew that it was over-priced.

Semantic expectations n n n Taraban & Mc. Celland, 1988 The couple admired the Semantic expectations n n n Taraban & Mc. Celland, 1988 The couple admired the house with a friend but knew that it was overpriced. The couple admired the house with a garden but knew that it was overpriced. The Non-MA structure may be favoured

What about spoken sentences? n All of the previous research focused on reading, what What about spoken sentences? n All of the previous research focused on reading, what about parsing of speech? n Methodological limits – ear analog of eye-movements not well developed n n Auditory moving window Reading while listening Looking at a scene while listening Some research on use of intonation

Intonation as a cue A: I’d like to fly to Davenport, Iowa on TWA. Intonation as a cue A: I’d like to fly to Davenport, Iowa on TWA. B: TWA doesn’t fly there. . . B 1: They fly to Des Moines. B 2: They fly to Des Moines.

Chunking, or “phrasing” A 1: I met Mary and Elena’s mother at the mall Chunking, or “phrasing” A 1: I met Mary and Elena’s mother at the mall yesterday. A 2: I met Mary and Elena’s mother at the mall yesterday.

Phrasing can disambiguate Mary & Elena’s mother mall I met Mary and Elena’s mother Phrasing can disambiguate Mary & Elena’s mother mall I met Mary and Elena’s mother at the mall yesterday One intonation phrase with relatively flat overall pitch range.

Phrasing can disambiguate Elena’s mother Mary mall I met Mary and Elena’s mother at Phrasing can disambiguate Elena’s mother Mary mall I met Mary and Elena’s mother at the mall yesterday Separate phrases, with expanded pitch movements.

Summing up n Is ambiguity resolution a problem in real life? n n Yes Summing up n Is ambiguity resolution a problem in real life? n n Yes (Try to think of a sentence that isn’t partially ambiguous) Many factors might influence the process of making sense of a string of words. (e. g. syntax, semantics, context, intonation, cooccurrence of words, frequency of usage, …)