Word Recognition Sereno 4 04 How long does it

Word Recognition (Sereno, 4/04) How long does it take to recognise a visual word? – What is meant by “recognition” or “lexical access”? – Can lexical access be accurately measured? – What factors affect lexical access and when? The “magic moment” (Balota, 1990) of lexical access: “At this moment, presumably there is recognition that the stimulus is a word, and access of other information (such as the meaning of the word, its syntactic class, its sound, and its spelling) would be rapid if not immediate. ” (Pollatsek & Rayner, 1990)

Background: Basic Units of Language A. ~5, 000 languages phonemes morphemes sentences conversations (sounds) & words B. Phonemes = elementary sounds of speech • phonemes are not letters. . . to, too, two, through, threw, shoe, clue, view • vowel & consonant phonemes • 11 -144 phonemes in any given language English has ~ 40; Hawaiian has ~16 • combining phonemes is rule-governed

Wordness: For each row of 3 possible new words, which one will probably never make it : ( blick splunge rlight sbarm wumple turl mancer nserht crelurious inther iwhucr neen shace fring ngout

Basic Units of Language C. Morphemes = smallest meaningful unit of lang. • can be a word, word stem, or affix (prefix, suffix) help, love “free” { word: word stem: spir, ceive, duce “bound” prefix/suffix: re-, dis-, un- / -less, -ful, -er • derivational & inflectional morphemes derivational – change the grammatical class V + -able = Adj (adorable, believable) V + -er = N (singer, runner) inflectional – grammatical markers V + -ed = past tense (walked) N + -s = plural (cows) {

Basic Units of Language C. Words • Content vs. function (open- vs. closed-class) words Content words = carry the main meaning nouns, verbs, adjectives, adverbs Function words = grammatical words articles (a, the, this), conjunctions (and, but), prepositions (in, above) Psychological reality of the content-function word distinction in aphasia selective impairment of content (Wernicke’s) or function words (Broca’s aphasia) • Cattell (1886) & Stroop (1925)

Word superiority effect (Cattell, 1886) – Reicher (1969); Wheeler (1970) – tachistoscopic presentation word --- d d k – more accurate identification of the letter when stimulus is a word – pseudoword superiorty effect

NAME THE COLOUR OF THE INK GREEN RED BLUE BLACK BLUE RED GREEN BLACK RED BLUE GREEN BLACK GREEN BLUE BLACK RED BLUE GREEN

Basic Units of Language C. Words • Ambiguity 1 word form, but 2 (or more) word meanings hs Ex: bank (N-N, “money” vs. “river”) p ra watch (N-V, “clock” vs. “look”) og om bass (N-N, “guitar” vs. “fish”) h 2 word forms, but 1 pronunciation es Ex: sail/sale, right/write n ho op om Generally unaware of ambiguity. . . h even though it is quite pervasive even though it affects behaviour (RT, etc)

Basic Units of Language D. Sentences • Syntax = the rule-governed system for grouping words together into phrases and sentences • Sentences introduce a concept that they are about, the subject (or noun phrase), and then propose something about that concept, the predicate (or verb phrase). Ex: “The boy hit the ball. ” doer subject act done-to (thematic roles) predicate

Basic Units of Language D. Sentences • Same deep structure, different surface structure “The boy hit the ball. ” (active) “The ball was hit by the ball. ” (passive) • Same surface structure, different deep structure [The French bottle]NP [smells. ]VP “The French bottle smells. ” [The French]NP [bottle smells. ]VP THEY are boring. “Visiting relatives can be boring. ” VISITING THEM is boring. cf. ambig. figures in perception: 1 form, 2 interpretations

Necker cube

Headlines New obesity study looks for larger test group Reagan wins on budget, but more lies ahead Man struck by lightening faces battery charge Enraged Cow Injures Farmer with Axe Milk Drinkers Are Turning to Powder Local High School Dropouts Cut in Half British Left Waffles on Falklands Dealers Will Hear Car Talk at Noon Miners Refuse to Work after Death Beating Witness Provides Names Squad Helps Dog Bite Victim Kids Make Nutritious Snacks

Headlines Stolen Painting Found by Tree Prostitutes Appeal to Pope Red Tape Holds up Bridge Deer Kill 17, 000 Teenage Prostitution Problem is Mounting Child Stool Great for Use in Garden Shouting Match Ends Teacher’s Hearing Man Robs then Kills Himself Lung Cancer in Women Mushrooms Mondale’s Offensive Looks Hard to Beat Tuna Biting off Washington Coast Chinese Apeman Dated

Basic Units of Language D. Sentences • Syntactic ambiguities “She hit the boy with the big stick. ” “She hit the boy with the runny nose. ” Interpretation depends on structural preferences (certain constructions used more often, favoured), as well as the prior discourse context.

Word Recognition • • • (Sereno, 4/04) Measures Components Models Eye movements (EMs) Event-related potentials (ERPs) Word frequency & lexical ambiguity

Measures • Standard behavioural techniques – lexical decision, naming, categorisation; also RSVP, self-paced reading – priming, masking, lateralised presentation – Donders (1868): subtractive method • assumes strictly serial stages of processing • additive vs. interactive effects – automatic vs. unconscious exogenous bottom-up benefit strategic (Posner & Snyder, 1975) controlled endogenous top-down cost & benefit

Measures • Eye movements • Neuroimaging – “Electrical”: EEG, MEG – “Blood flow”: PET, f. MRI

TASK MEASURE TIME RES. various word tasks “electrical” imaging: EEG, MEG ms-by-ms Normal reading fixation duration (as well as location and sequence of EMs) GOOD ~250 ms Standard word recognition paradigms (± priming, ± masking): naming lexical decision categorisation various word tasks RT ~500 ms ~600 ms ~800 ms “blood flow” imaging: f. MRI, PET seconds POOR

Components • Orthography of language – English vs. Hebrew or Japanese • Language skill – beginning (novice) vs. skilled (expert) reader – easy vs. difficult text

Components • Intraword variables – word-initial bi/tri-grams – spelling-to-sound regularity – neighborhood consistency – morphemes • prefix vs. pseudoprefix • compound vs. pseudocompound clown vs. dwarf hint vs. pint made vs. gave remind vs. relish cowboy vs. carpet

Components • Word variables – word length – word frequency – Ao. A – ambiguity – syntactic class – concreteness – affective tone – etc. duke vs. fisherman student vs. steward dinosaur vs. university bank vs. edge, brim open vs. closed; A, N, V tree vs. idea love vs. farm vs. fire

Components • Extraword variables – contextual predictability The person saw the. . . moustache. The barber trimmed the. . . – syntactic complexity Mary took the book. *Mary took the book was good. Mary knew the book was good. *Mary hoped the book was good. – discourse factors (anaphora, elaborative inferences) He assaulted her with his weapon. . knife. . . stabbed

Models • Dual-route account (Coltheart, 1978) semantics phonology Indirect route (assembled) Direct route (addressed) orthography

Models • Interactive (Morton, 1969; Seidenberg & Mc. Clelland, 1989) context meaning orthography MAKE phonology /m A k/

Models • Modular (Forster, 1979; Fodor, 1983) Message processor Syntactic processor General Problem Solver Lexical processor input features decision output

Models • Hybrid – 2 -stage: generate candidate set selection – (Becker & Killion; Norris; Potter)

Word Recognition • • • (Sereno, 4/04) Measures Components Models Eye movements (EMs) Event-related potentials (ERPs) Word frequency & lexical ambiguity

TASK MEASURE TIME RES. various word tasks “electrical” imaging: EEG, MEG ms-by-ms Normal reading fixation duration (as well as location and sequence of EMs) GOOD ~250 ms Standard word recognition paradigms (± priming, ± masking): naming lexical decision categorisation various word tasks RT ~500 ms ~600 ms ~800 ms “blood flow” imaging: f. MRI, PET seconds POOR

Tools of choice: • Recording eye movements in reading • Recording ERPs in language tasks

Eye Movements (EMs) Best on-line measure of visual word recognition in the context of normal reading: • Fast (avg fixation time ≈ 250 ms) • Ecologically valid task • Eye-mind span is tight

Number of trials EEG 1 2 4 8 P 1 P 300 ERP 16 N 1 N 400

ERPs Best real-time measure of brain activity associated with the perceptual and cognitive processing of words: • Continuous ms-by-ms record of events • Early, exogenous components (before 200 ms) should reflect lexical processing

(Sereno & Rayner, Trends in Cognitive Sciences, 2003)

High-density ERP Analysis: A case of “too many notes”? DIV IO RS E N

High-density ERP Analysis: Typical approaches for space & time • Pick ‘n choose favourite electrode and ERP component

High-density ERP Analysis: Typical approaches for space & time • Pick ‘n choose favourite electrode and ERP component • Hunt down where/when the effect is strongest and gather data from those electrodes/time window

High-density ERP Analysis: Typical approaches for space & time • Pick ‘n choose favourite electrode and ERP component • Hunt down where/when the effect is strongest and gather data from those electrodes/time window • Procrustean regions analysis (turtle shell) or series of pre-set time windows (eg, 50, 100, 200 ms)

High-density ERP Analysis: Typical approaches for space & time • Pick ‘n choose favourite electrode and ERP component • Hunt down where/when the effect is strongest and gather data from those electrodes/time window • Procrustean regions analysis (turtle shell) or series of pre-set time windows (eg, 50, 100, 200 ms) • Spatial and/or temporal principal component analysis (PCA)

Scalp topography of the N 1 @ 132 -192 ms SF 1 loadings Voltages (Sereno, Brewer, & O’Donnell, Psychological Science, 2003)

Word frequency • Word frequency effect represents the differential response to commonly used high-frequency (HF) words versus low-frequency (LF) words that occur much less often. • Presence of word frequency effects is used as a marker of successful word recognition or lexical access.

490 ms 553 ms 259 ms 275 ms 280 ms 293 ms (Sereno & Rayner, Trends in Cognitive Sciences, 2003)

Comparative EM & ERP studies • Word frequency (LF) rump The sore on Tam-Tam’s (HF) back was swollen. • Word frequency X Context (Neutral context) To our surprise we saw a (Biasing context) Flying to its nest was a hawk. (LF) (Neutral context) She looked over the (Biasing context) She read the new (HF) book. (Sereno, Rayner, & Posner, Neuro. Report, 1998) (Sereno & Rayner, Perception & Psychophysics, 2000) (Sereno, Brewer, & O’Donnell, Psychological Science, 2003)

More comparative EM & ERP studies • Word frequency X Orthographic regularity (LF-Reg) cask Mike wasted the whole (LF-Exc) pint (HF-Reg) week (HF-Exc) hour and then regretted it. • Lexical ambiguity X Context (Neutral context) James peered over at the bank. (Biasing context) The mud was deep along the (Sereno, Pacht, & Rayner, Psychological Science, 1992) (Sereno, JEP: Learning, Memory, and Cognition, 1995) (Sereno, Rayner, & Posner, Neuro. Report, 1998) (Sereno & Rayner, Perception & Psychophysics, 2000) (Sereno, Brewer, & O’Donnell, Psychological Science, 2003)

(Sereno & Rayner, Trends in Cognitive Sciences, 2003)

Lexical Ambiguity Resolution • Interactive position – Access is selective: Context guides access towards the appropriate sense of an ambiguous word; while both senses may be initially activated, only the contextually appropriate sense is fully accessed. • Modular position – Access is exhaustive: All meanings of ambiguous words are automatically accessed; context cannot directly affect lexical processing, but instead operates on the output of the lexical processor to select the appropriate sense.

Ambiguity: Cross-modal priming • Paradigm: Auditory context Aud amb Visual prime target “The building was infested with BUGS” ANT SPY SEW “and it…” (Swinney, 1979) • Results: – In general, support the modularity of lexical processing.

Ambiguity: ERP unimodal priming • Paradigm: – The only ERP ambiguity study employed a unimodal (visual) version of the cross-modal paradigm (Van Petten & Kutas, 1987). – Measured ERPs to targets that followed presentation of the ambiguous word prime. • Results: – Support an interactive account of lexical processing.

Ambiguity: EM “fast priming” • Paradigm: – Measured fixation duration on targets that followed presentation of the “fast” ambiguous word prime across various context conditions. . .

Ambiguity: EM “fast priming” Priming Effect ~30 ms n. s. (Sereno, JEP: LMC, 1995)

Ambiguity: EM “fast priming” • Results: Support a modified interactive account of lexical processing - “reordered access” - in which both (1) meaning frequency (Dom vs. Sub) (2) prior context affect access speed. Specifically, (1) Alternative meanings become activated in order of their meaning frequency (2) Context can “boost” the activation of one of the meanings, possibly reordering access procedures

Ambiguity: EM normal reading • Paradigm: Ambiguous (subordinate) The mud was deep along the bank. . . LF The mud was deep along the brim. . . HF The mud was deep along the edge. . . • Results: Fixation time: “Spillover” time: Amb = LF > HF Amb > LF > HF Support modified interactive account of access: “reordered access”. (Sereno, Pacht, & Rayner, Psych Sci, 1992)

Critique of methods: Cross-modal priming • RT (lexical decision, naming) – Slow (500 -900 ms) compared to speed of lexical access (~100 -200 ms); susceptible to response bias. • Secondary (indirect) measure – Effects of context on ambiguous word gauged by priming effects on target downstream.

Critique of methods: ERP unimodal priming • Time-locked averages of the EEG – Ms-by-ms voltage fluctuations reflect processing in real time. • Secondary (indirect) measure – Effects of context on ambiguous word gauged by priming effects on target downstream.

Critique of methods: EM “fast priming” • Fixation time – Relatively fast (~375 ms) and on-line, but can reflect lexical and post-lexical integration effects. • Secondary (indirect) measure – Although much quicker time course than crossmodal or ERP unimodal, effects of context on ambiguous word still gauged by priming effects on target downstream

Critique of methods: EM normal reading • Fixation time – Fast (~250 ms) and on-line, but can reflect lexical and post-lexical integration effects. • Primary (direct) measure – Effects of context on ambiguous word gauged by comparing its fixation time to control word.

ERP Ambiguity Experiment • Biased ambiguous words were presented in neutral and biasing contexts in a word-by-word sentence reading paradigm. Biasing contexts always instantiated the subordinate sense. • ERPs on the ambiguous words, themselves, were measured. • ERPs to ambiguous words were then directly compared to ERPs to unambiguous control words. • Control words - matched either to the dominant (HF) sense of the ambiguous word or to the contextually instantiated subordinate (LF) sense of the ambiguous word - were presented in neutral and biasing contexts. • Comparisons across conditions were made at an early, lexical stage of processing (N 1, 132 -192 ms).

Example Stimuli

Scalp topography of the N 1 @ 132 -192 ms SF 1 loadings Voltages

Scalp topography of the N 1 @ 132 -192 ms SF 1 loadings Voltages ± 0. 7 factor loading contours

N 1 @ 132 -192 ms Factor scores for SF 1 Voltages (electrodes with SF 1 loading > +0. 7 or SF 1 loading < -0. 7)

(Sereno & Rayner, Trends in Cognitive Sciences, 2003)

Summary • From the EM record, we can infer when lexical processing should occur (~100 -200 ms). • From the ERP record, we can see when certain differences first appear in real time. • Effects of word frequency as well as context initially appear very early in the ERP record (N 1 @ 132 ms post-stimulus). • We can begin to establish a realistic time-line of word recognition in reading.

ERP Word Recognition Stimuli Words LF HF Reg cask time Pseudo. Words Exc pint hour Consonant Strings welf fhvr Results Lexicality Frequency Regularity W vs PW vs CS LF vs HF LF Reg vs LF Exc P 1 @ 100 -132 ms N 1 @ 132 -164 ms P 2 @ 164 -196 ms (Sereno, Rayner, & Posner, Neuro. Report, 1998)

Lexicality Effects: P 1 (100 -132 ms) CS-W PW-W CS-PW p< (Sereno, Rayner, & Posner, Neuro. Report, 1998)

Frequency Effects: N 1 (132 -164 ms) LF-HF (Sereno, Rayner, & Posner, Neuro. Report, 1998)

Regularity Effects: P 2 (164 -196 ms) LF Exc -LF Reg Ss with RT effect Ss with no RT effect (Sereno, Rayner, & Posner, Neuro. Report, 1998)

Scalp topography of the N 1 @ 132 -192 ms SF 1 loadings Voltages ± 0. 7 factor loading contours (Sereno, Brewer, & O’Donnell, Psychologcial Science, 2003)

Emotion words Arousal Valence + ve Lo peace Hi love – ve bored fire Neutral controls: hotel, farm

Questions of interest: • How fast are words recognised? • What factors affect lexical access? • How early do these factors operate?

Models • Interactive vs. Modular – Logogen (Morton, 1969) – PDP (Seidenberg & Mc. Clelland, 1989) • Read-out – Search (Forster & Bendall, 19 xx; ) • Hybrid – 2 -stage: generate candidate set selection – Becker & Killion, 19 xx; Norris, 1984; Potter

Jokes Q: Why are camels called “ships of the desert? ” A: Because they’re always filled with Arab sea-men. Q: Would you prefer roses on your piano or tulips on your organ? Q: What’s the difference between a rolling stone and a Scotsman? A: A Rolling Stone says “Hey you get off of my cloud!” and a Scotsman says “Hey Mc. Leod get off my ewe!” Q: Why is men’s ‘sea-men’ white and their urine yellow? A: So they can tell if they’re coming or going.

Current Directions • Emotion word processing • Contextual constraint

ERP Ambiguity Experiment • Design/Stimuli 6 experimental conditions: Word Type LF HF Amb X Context Neutral Biasing Word specifications: LF = 6 per million HF = 60 per million Amb = 63 per million (Dominant sense = 89 % Subordinate sense = 9%)

Spatial Principal Components Analysis • Sample-by-sample voltage data at all 129 electrodes for all 14 Ss in all 6 conditions were submitted to a spatial PCA with a Quartimax rotation (cf. Spencer, Dien, & Donchin, 1998). • The first Spatial Factor (SF 1) accounted for a high degree of the variance (44%). • 3(Word Type) x 2(Context) ANOVA was performed on the factor scores.