Dr. Zenzi M. Griffin
| |
Cognition & Communication Lab |
| |
Dr. Zenzi M. Griffin |
|
Research Courses |
Speaker's Eye Movements:
Each
movie shows a stimulus picture with places
where the eye rested (i.e., fixations) plotted as circles. The longer
the
fixation, the larger the circle gets. The current fixation is green and
previous ones are blue. Most
examples start with speech and eye
movements
played back at half speed followed by a playback at fullspeed. The
timing
used to create these examples is not precise, but they give a good feel
for the timing of speech relative to eye movements. These movies were
created from the raw data recorded in the experiments but do NOT
constitute the raw
data.
Griffin, Z. M., & Oppenheimer, D. (2006). Speakers gaze at objects while preparing intentionally inaccurate labels for them. Journal of Experimental Psychology: Learning, Memory, and Cognition, 32(4), 943-948. When describing scenes, speakers gaze at objects while
preparing their names (Z. M. Griffin & K. Bock, 2000). In this
study, the authors investigated whether gazes to referents occurred in
the absence of a correspondence between visual features and word
meaning. Speakers gazed significantly longer at objects before
intentionally labeling them inaccurately with the names of similar
things (e.g., calling a horse a dog) than when labeling them
accurately. This held for grammatical subjects and objects as well as
agents and patients. Moreover, the time spent gazing at a referent
before labeling it with a novel word or accurate name was similar and
decreased as speakers gained experience using the novel word. These
results suggest that visual attention in speaking may be directed
toward referents in the absence of any association between their visual
forms and the words used to talk about them.
Griffin, Z. M. (2004). The eyes are right when the mouth is
wrong. Psychological Science, 15(12),
814-821. [preprint
pdf]
Griffin, Z. M. (2003). A reversed word length effect in coordinating the preparation and articulation of words in speaking. Psychonomic Bulletin and Review, 10(3), 603-609. [preprint pdf] When describing scenes, speakers tend to select their nouns in the second before saying them rather than preparing and buffering them further in advance (Griffin & Bock, 2000). To test the limits of this last-second preparation, speakers were asked to name two objects without pausing between them. There was not enough time to prepare the second object's name while saying the first. The length of the first name determined the extent to which speakers prepared the second one before speaking. When the first word was monosyllabic (e.g., wig), speakers spent more time preparing the second word (carrot) before speaking than they did when the first was multisyllabic (windmill). Preparation differences were reflected in speech latencies and eye movements. The results suggest that speakers are able to estimate the time available for preparing words during speech and to use this information to increase fluency while minimizing word buffering.
Griffin, Z. M. (2001). Gaze durations during speech reflect word selection and phonological encoding. Cognition, 82, B1-B14. [preprint pdf] Speakers produced the sentence frame The A and the B are above the C to describe three pictured objects while their eye movements were monitored. Object B or C varied in codability (the number of alternative names for it) and in the frequency of its dominant name. Codability is known to affect speed of word selection, and word frequency, speed to retrieve a word's pronunciation (phonological encoding). Speakers gazed longer at lower codability and lower frequency objects before naming them. However, the codability and frequency of B and C did not affect when speakers began naming A, even when utterances were perfectly fluent. The results indicate that speakers began "The A" once they had a name prepared for A, before selecting names for B and C. Similar gaze patterns during less constrained scene description tasks in other studies suggest that speakers often incrementally select and phonologically encode nouns in fluent utterances. Griffin, Z. M., & Bock, K. (2000). What the eyes say about speaking. Psychological Science, 11(4), 274-279. To study the time course of sentence formulation, we monitored the eye movements of speakers as they described simple events. The similarity between speakers' initial eye movements and those of observers performing a non-verbal event comprehension task suggested that response-relevant information was rapidly extracted from scenes, allowing speakers to select grammatical subjects based on comprehended events rather than salience. When speaking extemporaneously, speakers began fixating pictured elements less than a second before naming them within their descriptions, consistent with incremental lexical encoding. Fixation trajectories anticipated the order of mention despite changes in picture orientation, in who-did-what-to-whom, and in sentence structure. The results support Wundt's theory of sentence production.
Griffin, Z. M. (1998). What the eye says about sentence planning. Unpublished doctoral dissertation, University of Illinois at Urbana-Champaign, IL. Three experiments used the eye movements made by speakers across pictured events to investigate the time course of sentence planning and execution. The first experiment explored the relationship between perceptually encoding actors in simple events and the temporal sequence of linguistic expressions referring to them in speech. When possible, speakers appeared to begin fluent descriptions soon after fixating every actor in an event briefly, thereby comprehending the event. The time spent looking at actors before and during speech indicated that, when speaking extemporaneously, the sentential [sic] subject was lexically encoded before speech onset whereas the direct object was lexically encoded during speech. The subsequent experiments more rigorously tested this difference. In Experiment 2, the event roles of the actors in each picture were exchanged while retaining as much other detail as possible. For one picture set, switching the agent of a depicted action changed the identity of the actor encoded as subject. This in turn caused a change in eye movement patterns across the actors. For the other picture set, switching event roles did not affect the order in which actors were mentioned, but required a different syntactic structure to accommodate the change. This exchange of event roles did not substantially alter the time course of eye movements across actors, further indicating that order of mention determined the distribution of fixation times across time. In Experiment 3, speakers were provided with verbs to use in their descriptions. Because the verbs preferred different mappings between event roles and grammatical roles, the sequence of eye movements changed accordingly. Furthermore, being provided with a verb allowed speakers to start speaking earlier than they would otherwise, fixating fewer actors in the event, and looking at the named actors for less time before speaking. The results demonstrate the utility of eye movement data in the study of sentence processing and provide new evidence about the time course of lexically encoding nouns in sentences.
|
|
|
Quicktime movies of eye movements
and speech
