APA Style Journal 0.

1

Copyright 2002 by Frouke Hermens

The production of multiple words utterances
Frouke Hermens
Department of Biophysics University of Nijmegen

Antje S. Meyer
Department of Psychology University of Birmingham

Willem J.M. Levelt
Max Planck Institute for Psycholinguistics Nijmegen

In two series of experiments the retrieval of multiple content words from the mental lexicon is investigated. In the first series of experiments participants were asked to name pictures presented simultaneously on a computer screen. The names of the first two pictures were either semantically related, phonologically related, or unrelated. Speech onset latencies and viewing times for the first two objects were measured. No evidence of parallel retrieval was found in these three experiments. In an additional set of experiments two pictures were presented one by one at the same location on the computer screen. As in the first series of experiments the name of the two pictures could be semantically related, phonologically related, or unrelated. Effects of semantic relatedness were found when both pictures were named in the reversed order of presentation, when the second picture was named, and when the second picture was classified as an object or a nonobject. Because semantically related objects tend to be similar in shape participants were asked to rate the visual similarity of object pairs. No effects of visual similarity ratings were found on response times. All together the results suggest that object names are retrieved from the mental lexicon as sequentially as possible. The observed errors suggest that they only occur when people fail to retrieve words in a serial manner.

Introduction
Experiments investigating the retrieval of words from the mental lexicon often make use of single object naming tasks. Less research has been concerned with the production of multiple content words utterances. Earlier research on the retrieval of multiple content words from the mental lexicon has focussed on the errors made in spontaneous speech production. For instance, in a study by Dell (1986) errors observed in natural speech production were used to construct a model. Utterances like ’they key question’ led to the conclusion that the retrieval of multiple content words is carried out partially simultaneously. To have an effect on the first word to be produced the retrieval of the second word must start before the retrieval of the first has finished. Additional support for parallel retrieval of words was found by Martin, Weisberg, and Saffran (1995). In their experiments participants were asked to describe scenes containing multiple objects. Mixed errors (both semantic and phonological in nature) were found more often than would be predicted on the basis of the counts of purely semantic and purely phonological errors. In two series of experiments we tried to find additional evidence for the hypothesis that content words to be produced in a single utterance are retrieved in parallel. Instead of studying errors we focussed on speech onset latencies and viewing times. From picture-word interference experiments

(Glaser & D¨ ngelhoff, 1984; Meyer & Schriefers, 1991) it is u know that the production of picture names is delayed if a semantically related word is presented some time before speech onset. In contrast, the presentation of a phonologically related word quickens speech onset. These effects were used to study the time-course of multiple content words retrieval using speech onset latencies. In the first series of experiments pictures were presented simultaneously on a computer screen. One task involved participants to name the pictures using utterances like ’apple and pear and house’. Another group of participants was asked to include determiners in the object description resulting in utterances like ’the apple, the pear, and the house’. A third task was to name two pictures in an utterance like ’the apple and the pear’ and then to make a decision on the shape of a third object. In all three tasks the first two objects names were chosen either to be semantically related, phonologically related, or unrelated. If the retrieval of the first two words is carried in a parallel fasion, object names are expected to influence each other, resulting in different speech onset latencies for related and unrelated object names. Two utterance formats were tested to see what effect the production of determiners has on speech onset latencies and speech errors. Since in the first two naming tasks participants were found to have difficulties in fluently naming the three objects the number of objects to be named was reduced to two in the third naming task. In experiments by Meyer, Sleiderlink, and Levelt (1998), Van der Meulen, Meyer, and Levelt (2001), and Meyer and Van der Meulen (2001) fixation durations were found to be
1

We would like to thank Eva Belke for running Experiment 1c.

2

HERMENS, MEYER, AND LEVELT

related to word retrieval durations. Because of these findings eye movements of participants were recorded during the three picture naming tasks. If retrieval occurs in parallel similar effects are expected for naming latencies and fixation durations. In the second series of experiments we used a setup which would increase the likelihood of parallel processing of object names. Pictures were presented one by one at the same location of the computer screen. In the first experiment participants were asked to name the pictures in the reversed order of presentation. For example, if first a picture of a tree was presented followed by a picture of a house, the task of the participant would be to produce the phrase ‘the house and the tree’. As in the first series of experiments the relatedness of the picture names was varied. If participants (partially) retrieve the name of the first object before retrieving the second object’s name, effects of relatedness are expected on speech onset latencies. In addition to relatedness the presentation duration of the first picture was varied. If as many stages of word retrieval as possible are carried out upon presentation of the first picture, more interference is expected for longer presentation durations of the first picture. The next two experiments served as control experiments of reversed order naming experiment. In one experiment the so-called picture-picture priming effect was tested by asking participants to name just the secondly presented picture. Again the effect of relatedness and first picture presentation duration were investigated. In the second control experiment participants were asked to determine whether the secondly presented object was an existing object. In this experiment pictures were intermixed with pictures of nonobjects (see, Kroll & Potter, 1984) If effects found in the reversed order naming experiment involve just the retrieval of object names no effect of relatedness is expected in an object-nonobject decision task. Semantically related pictures tend to be visually related too. Therefore we asked a group of participants to judge the visual similarity of the semantically related object pairs. If visual similarity has an effect on response times a correlation is expected between visual similarity ratings and response times.

Experiment 1a
In the first experiment three pictures were presented simultaneously on a computer screen. Participants were asked to name the objects in utterances like ’tree, house, and table’.

On each trial participants were presented with three line drawings of objects, e.g. ’trompet’ (’trumpet’), ’viool’ (’violin’) and ‘boom’ (‘tree’), which they named in a noun phrase conjunction, such as ’Trompet, viool en boom’ (’Trumpet, violin and tree’). The first two objects of each triple were 80 objects selected from a pool used in earlier experiments on lexical access in language production (see Appendix A). It was known from these studies how the objects were usually named. Forty objects with monosyllabic names were combined into twenty phonologically related pairs (e.g., ’beer-been’ (’bear-leg’) or ’schaar-schaats’ (’scissors-skate’)). Their names shared the word-initial consonant or consonant cluster and the following vowel. The same forty objects were recombined into twenty phonologically unrelated pairs. The unrelated pairs were created out of two related pairs. For example, the pairs ’beer-been’ and ’schaar- schaats’ were transformed into ’schaar-been’ and ’beer-schaats’. The remaining forty objects were combined into twenty categorically related pairs such as ’raket-vliegtuig’ (’missileaeroplane’) and ’trompet-viool’ (’trumpet-violin’). These objects were also recombined into unrelated pairs. Results of picture- word interference experiments (e.g., Glaser & D¨ ngelhoff, 1984; Meyer & Schriefers, 1991) in which u speakers named pictures while they heard or saw distracter words to which no overt reaction was required, have shown that different types of semantically related distracters may differ in their effects. Associatively related distracters can facilitate target naming relative to unrelated distracters, whereas categorically related distracters may interfere with target naming. To avoid that categorical and associative effects cancel each other object pairs were created that were only categorically, but not associatively related. A pre-test was carried out to identify associatively related pairs. One group of 20 participants received a list of the names of the objects planned to appear as left objects in the main experiment, and another group of 20 participants received a list with the names of the right objects. The participants wrote down the first word that came to mind as they read each word. For no item the other member of the pair was given by more than four participants. To each of the pairs a third object was added with a name unrelated to the names of the first two objects. The added objects were drawn from the same set as the object pairs. Fifteen practice trials were created by making triplets consisting of objects from the experimental set. The names of the objects within each practice triplet were unrelated. For each participant ten practice trials were chosen at random. Design. The materials included 20 object triplets with semantically related names of the first two objects, 20 unrelated object triplets made up of the same objects (called ’semantically unrelated hereafter, which were unrelated in form as well), 20 object triplets with phonologically related names of the first two objects, and 20 phonologically unrelated ones. The experiment included four blocks of trials, in each of which 10 triplets of each of the four types were tested. In the first

Method
Participants. The experiment was carried out with sixteen participants drawn from the Max Planck Institute participant pool. All were native speakers of Dutch. Each participant was paid fl. 8.50 for their participation. Materials.

PRODUCTION OF MULTIPLE WORDS UTTERANCES

3

two blocks the first two pictures were presented as they are listed in Appendix A. In the other two blocks the presentation order of the first two pictures was reversed. All speakers were tested on all four blocks. The order of the blocks was counterbalanced across speakers. In the first block four practice items were presented, while in the other three blocks two practice items were used. Between the four blocks there were breaks which could be used for recalibrating the camera’s of the eye tracking device. Apparatus. The experiment was controlled by a standard PC. The pictures were presented on a ViewSonic 17PS screen. Speech onset latencies were measured using a voice-key. For the presentation of the stimuli the NESU experimental software available at the Max Planck Institute was used. Eye movements were monitored using an SMI EyeLinkHispeed 2D head-mounted eye tracking system (SensoMotoric Instruments GmbH, Teltow, Germany). Throughout the experiment, the position of the right eye was tracked with a sampling rate of 4 ms. The eye tracker’s spatial accuracy is better than 0.5 degree. Three thresholds were used to detect the onsets and offsets of saccades: motion (0.2 degrees), velocity (30 degrees/second), and acceleration (8000 degrees/second2). The duration of a fixation was the time period between two successive saccades. The position of a fixation was defined as the means of the x- and y- coordinates of the positions recorded during the fixation. Procedure. The participants were tested individually. They were seated in a quiet room approximately 65 cm in front of a monitor. They first received a booklet including drawings of the practice and experimental objects. The names of the objects were printed next to them. In a written instruction, that the participants received together with the picture booklet, they were told that they would later see triplets of objects which they should name, from left to right in a utterance like ’boom, huis, en boot’ (’tree, house and boat’). The instructions included a picture of an object triplet and an example of a triplet description. After the participants had read the instruction and studied the picture booklet, the head band of the eye-tracking system was mounted and the system was calibrated. For the calibration, a grid of three by three positions was used. During a calibration trial a fixation target appeared one by one, in random order, on each of these positions for one second. The participants were asked to fixate upon each target until the next target appeared. After the calibration trial, the estimated positions of the participant’s fixations and the distances from the fixation points were graphically displayed to the experimenter. Calibration was considered adequate if there was at least one fixation within 1.5 degrees of each fixation target. When calibration was inadequate, the procedure was repeated, sometimes after adjustment of the cameras. Successful calibration was followed by a validation trial. For the participants, this trial did not differ from the calibration trial, but the data collected during the validation trial were used to

estimate the participants’ gaze positions, and the error (i.e., the distance between the estimated gaze position and the target position) was measured. The validation was considered adequate if the average error was below 1 degree and the worst error below 1.5 degree. Depending on the result, the calibration and validation trials were repeated or the main part of the experiment started. Calibration and validation were repeated after each test block, when needed (because of, for example, movements of the participant). At the beginning of each test trial in the main experiment, a fixation point was presented in the center of the frame for the left object for 800 ms. Our earlier experiments, in which two objects were presented, had shown that on more than 90% of the trials the participants naming object pairs first looked at the left and then at the right object. This strong tendency to inspect the objects in the order of mention was reinforced by the presentation of the fixation point. Following a blank interval of 200 ms, the three objects were presented for 4000 ms, and the participant named the objects. After another blank interval of 800 ms the next trial began. The entire session took about 45 minutes. Data analysis. The semantically related pairs of object triplets and their controls comprised different pictures than the phonologically related pairs of object triples and the controls. Since we were not interested in comparisons between these item sets, we analyzed them separately using paired t-tests with speakers and items as random variables, yielding t 1 and t2 statistics, respectively. For the off-line analyses of the eye movements, graphical software was used that displayed for each trial the locations of the speaker’s fixations as dots superimposed upon the line drawing. The first step in the analyses was to classify the fixations as falling on the left or right object or elsewhere. A fixation was categorized as pertaining to an object when it lay inside the contours of the object or less than 1.5 degrees away from one of its outer contours. Next, the speakers’ gaze patterns were examined. At the beginning of each trial a fixation point had appeared at the location where the left object would be shown a little later. Consequently, the speakers usually fixated upon the left object at picture onset. In the cases in which participants didn’t look at the left object during the first fixation the data was removed from the analysis. Also the cases in which participants didn’t look at one of the other two objects were not included in the analysis. Viewing times were determined by subtracting intimes from outtimes, where intimes were set to zero when fixation onset of the object occurred before picture onset. Outtimes were defined as the onset of the saccade to the second object.

Results
Onset latencies longer than 1700 ms were removed from the data analysis, as were trials in which the voice-key was triggered by other sounds than speech onset and non-fluent

4

HERMENS, MEYER, AND LEVELT

Table 1 Mean response times, viewing times (both measured in ms), and error rates (in parenthesis) by participants observed in Experiment 1a. Speech Onset Latencies Semantic Phonological Related 915 (18%) 893 (24%) Unrelated 908 (18%) 907 (16%) Viewing Times 1st Object Semantic Phonological Related 523 484 Unrelated 520 502 Viewing Times 2nd Object Semantic Phonological Related 472 473 Unrelated 496 448

Table 3 Mean response times, viewing times (both in ms), and error rates (in parenthesis) by participants observed in Experiment 1b. Speech Onset Latencies Semantic Phonological Related 842 (24%) 842 (29%) Unrelated 825 (24%) 831 (23%) Viewing Times 1st Object Semantic Phonological Related 645 613 Unrelated 638 619 Viewing Times 2nd Object Semantic Phonological Related 608 618 Unrelated 642 595

utterances. Only trials in which the participant looking at both pictures were analyzed. Mean latencies, viewing times, and error rates for each of the conditions are shown in 1. No effect of semantic relatedness was found on speech onset latencies (both t’s 1), on viewing times of the first object (both t’s 1), or on viewing times of the second object (both t’s 1). Significant differences in the participant analysis were obtained for the phonologically related and unrelated conditions for speech onset latencies (t 1 ´15µ 2 58 p 0 021; t2 ´39µ 1 60 n.s.), and for viewing times for the first object (t1 ´15µ 2 72 p 0 016; t 2 ´39µ 1 94 n.s.). The effect on the viewing time of the second object was significant only in the item analysis (t 1 1 84 n.s.; t2 ´39µ 2 19 p 0 035).

instruction. The participants were now asked to produce sentences like ‘De gitaar en de viool en de kast’ (‘The guitar, and the violin, and the cupboard’) when seeing the objects of a guitar, a violin and a cupboard.

Method
Participants. Sixteen participants drawn from the Max Planck Institute subject pool participated in the experiment. The participants had not taken part in the first experiment. Materials, Design, Apparatus, Procedure and Data Analysis. Materials, design, apparatus, procedure, and data analysis were the same as in the first experiment. Participants were now asked to name the three objects using phrases phrases like: ’de boom en het huis en de boot’ (’the tree and the house and the boat’).

Discussion
No consistent effects of semantic relatedness were found on speech onset latencies or viewing times. Only small effects of phonological relatedness were found. No evidence was found for parallell retrieval of object names to be produced in one utterance. Participants often made errors while naming the objects (error rates ranging from 5% to 34%). Errors were often found to be semantic in nature. However, these semantic errors were equally likely in both the semantic and the unrelated condition. A sample of the observed errors is presented in Table 2. The semantic errors suggest that semantic information of multiple object names were available during the planning process.

Results
As in Experiment 1a incorrect voice-key triggers, incorrect namings, determiners errors, and non-fluent utterances were removed from the data analysis. In addition, response times longer than 1700 ms were not included in the analysis as were trials in which one of the objects was not looked at. Mean response times and viewing times are shown Table 3. As in the first experiment no effects of semantic relatedness were found on speech onset latencies and viewing times of the first object (all t’s 1). In addition, no effects of phonological relatedness (t 1 ´16µ 0 56 n.s.; t 2 ´39µ 1 00 n.s.) were found on speech onset latencies. Neither were there phonological effects on the viewing times for the both the first (both t’s 1) and the second object (t 1 1 71 n.s.; t2 ´39µ 1 23 n.s.). The only significant effect was a semantic facilitation effect on viewing times of the second object (t1 ´16µ 3 27 p 0 005; t 2 ´39µ 2 04 p 0 048). Participants made many errors in naming the objects (error rates per participants ranged from 9% to 45%). The pro-

Experiment 1b
In Experiment 1a participants were asked to name the objects without determiners. This caused the object names to be temporally close to each other in the utterance. This might have resulted in the large number of errors. To check this hypothesis and to replicate the findings of Experiment 1b, the same experiment was carried out again, with a different

PRODUCTION OF MULTIPLE WORDS UTTERANCES

5

Table 2 Target Kop (’cup’) Peer (’pear’) Kanon (’cannon’) Been (’leg’) Bureau (’desk’) Bijl (’axe’) Schaap (’sheep’) Trein (’train’) Hamer (’hammer’) Raam (’window’) Blik (’dustpan’) Kop (’cup’) Fluit (’flute’) Beitel (’chisel’) Kop (’cup’) Gitaar (’gitar’) Schaats (’skate’) Boer (’farmer’)

Observed kan (’can’) appel (’apple’) raket (’missile’) arm (’arm’) kast (’cupboard’) hark (’rake’) varken (’pig’) vliegtuig (’plane’) beitel (’chisel’) kast (’cupboard’) veger (’brush’) schotel (’plate’) pijp (’pipe’) bijl (’axe’) kom (’bowl’) geweer (’gun’) schaar (’scissors’) hark (’rake’)

A sample of errors observed in Experiment 1a. Error Classification Semantically related to the target Semantically related to the target Semantically related to the target Semantically related to the target Semantically related to the target Semantically related to the target Semantically related to the target Semantically related to the target Semantically related to the target Semantially related to the target Complement object of target Complement object of target Shape similar objects Semantically and phonologically related to the target Semantically and phonologiclly related to the target Semanticlaly related to the target Phonologically related to the target Part of the picture is named

duction of the correct determiners was found to be difficult (determiner error rates ranged from 1% to 9%).

Discussion
The results of Experiment 1b are similar to those of Experiment 1a: No consistent effects of relatedness are found on either response latencies or viewing times. Utterances containing determiners are initiated earlier, but the objects are looked at for a longer time. The longer viewing times can be explained by the longer utterance durations. Eye movements are known to be related to the naming durations of the utterance. Longer viewing times are expected because it takes additional time to pronounce the determiner.

Materials. The items of Experiments 1a were used together with pictures of a cross (’¢’) and a plus sign (’+’). Only the first two objects of each triplet were presented to the participant on each trial. Design. The design of the experiment was similar to the ones used in Experiments 1a and 1b. Instead of a third object either a plus sign or a cross was added to an object pair. The third object (either a cross or a plus sign) was selected at random for each of the trials. Crosses were used more often (80% of the time) than plus signs because these didn’t require a response. Apparatus. The same equipment was used as in Experiments 1a and 1b. Participants responded to the plus signs by pressing a button placed on top of a bar held in the right hand. Procedure. Two objects and either a cross or a plus sign were presented to the participants. The first object was shown in the upper left corner, the second object in the upper right corner and the sign in the bottom center of the screen. Participants were instructed to first name the two objects in a fluent utterance like: ’de boom en het huis’ (’the tree and the house’), followed by a button press if the object at the bottom of the screen was a plus. The remaining of the procedure was the same as in Experiments 1a and 1b. Results. Response times slower than 400 ms and longer than 1350 ms were removed from the analysis. Trials in which a speech error occurred, or an incorrect button press had taken place

Experiment 1c
In both experiments in which three objects had to be named participants made quite some errors while naming the objects. To reduce the number of errors a new method was introduced for measuring viewing times of the second object. Instead of presenting a third object that had to be named, together with the two objects either a cross or a plus sign was shown in the bottom of the screen. The task of the participants was to name first the two objects in a fluent utterance followed by a decision task on the object in the bottom on the screen. When they saw a plus sign (’+’), a button had to be pressed, else no response was expected.

Method
Participants. The experiment was run with twelve participants selected from the Max Planck Institute subject pool. Participants were selected that had not participated in one of the previous experiments.

6

HERMENS, MEYER, AND LEVELT

Table 4 Mean response times, viewing times (both measured in ms), and error rates (in parenthesis) by participants observed in Experiment 1c. Speech Onset Latencies Semantic Phonological Related 796 (5 %) 764 (10 %) Unrelated 780 (6 %) 765 (7 %) Viewing Times 1st Object Semantic Phonological Related 562 540 Unrelated 557 535 Viewing Times 2nd Object Semantic Phonological Related 644 644 Unrelated 668 609

Experiment 2a
In Experiment 2a objects were presented one by one to have participants retrieve information of the second word to be produced before starting retrieving the name of the other object. Participants were asked to produce an utterance in which the second object to be presented was named first. This way partial parallel processing of object names made more likely, yielding additional information about what happens if related object names are processed simultaneously.

Method
Participants. Twenty-six participants from the Max Planck Institute participant pool took part in the experiment. None of the participants had participated in one of the first three experiments. All participants were native speakers of Dutch, and had normal or corrected-to-normal vision. They were paid fl. 8.50 for their participation. Response time data of six participants were not analyzed because of the large number of errors these participants made. Materials. The same objects pairs were used as in Experiment 1c. Because two pictures were often named incorrectly in the first series of experiments, they were replaced by others. The picture of the violin was replaced by one of a piano, and the ship was replaced by a drill. Design. Each object pair was presented four times in the experiment, resulting in a total of 160 trials. Object pairs were presented in two orders. Two presentation durations of the first object were used: 150ms and 350ms. Apparatus. The experiment was controlled by a standard PC. Stimuli were presented on a 15 inch computer screen. Participants were sitting at a distance of about 50 cm from the computer screen. Speech onset latencies were recorded using a voicekey. Procedure. Participants read in a written instruction that they would see two pictures presented one at a time on the computer screen. Their task was to name both pictures in a single utterance naming the last picture first. Utterances had to be of the format ’de slak en het glas’ (’the snail and the glass’), containing determiners and the word ’and’. Participants received a picture booklet containing all pictures together with their names. They were asked to study this picture booklet carefully. At the beginning of each trial, a fixation point was shown at the center of the screen for 600 ms, followed by a blank interval of 200 ms. Then the first picture was presented for either 150 ms or 350 ms (the short and the long presentation duration condition), followed by a blank screen for 50 ms. The second picture was presented for 3000 ms. During the presentation of the second participants tried to name both

were also removed from the analysis. Only cases in which both objects were looked at were included in the analysis. Mean speech onset latencies, viewing times of the first, and of the second object are shown in Table 4. A signficant 16 ms inhibition effect of semantic relatedness on onset latencies was found t1 ´11µ 2 38 p 0 037; t 2 ´39µ 2 13 p 0 040). All the other effects were not significant.

Discussion
No consistent effects of relatedness were obtained, as in Experiments 1a and 1b. The procedure to reduce the number of speech errors was found to be succesfull. In Experiment 1b percentages correct responses ranged from 55 % to 91 %. In Experiment 3 the percentage correct ranged from 90 % to 97 %, which is a clear improvement. However the change in task from Experiment 1b to Experiment 1c did not change the effects of semantic and phonological relatedness. To summarize the results of Experiments 1a through 1c, no consistent effects of phonological or semantic relatedness were found in experiments in which either two or three objects had to be named in a single utterance. If three objects have to be named more errors were made than if two objects were named followed by a decision task. If errors were made, these errors were often found to be semantic in nature. For example, if a banana, an apple, and a house had to be named, it was more likely that participants would substitute the banana with a pear than with another object name. However, semantic errors were not observed more frequently in the semantically related than in the unrelated condition. The first three experiments suggest that people try to plan an utterance as sequentially as possible. This conclusion is supported by the pattern of eye movements: Each object is looked at one by one as if objects are pointed at by the eyes when being named.

PRODUCTION OF MULTIPLE WORDS UTTERANCES

7

Table 5 Mean response times (in ms) and error rates (in parenthesis) by participants observed in Experiment 2a. 150 ms presentation duration Semantic Phonological Related 855 (21%) 835 (23%) Unrelated 890 (21%) 839 (20%) 350 ms presentation duration Semantic Phonological Related 797 (19%) 787 (21%) Unrelated 819 (17%) 784 (17%)

pictures as accurately as possible, initiating their utterances as fast as possible. The following trial started 550 ms after picture offset of the previous trial. After each 40 trials a break was included.

few phonological substitution errors are found. Even when participants had more time to process the first picture before starting processing the second picture no effects of phonological relatedness were found. However, the effect of presentation duration within the two relatedness conditions was significant. This suggests that some additional processing, not phonological in nature, takes place if the first picture is presented for a longer time. A semantic facilitation effect was found. In picture word interference experiments the effect is often inhibitory. The inhibition found in picture word interference experiments led to the conclusion that in the semantic processing stage alternatives compete for selection. However, in Experiment 2a both objects had to be named. Therefore none of the active object names had to be deactivated, and a facilitation effect is expected.

Experiment 2b
In Experiment 2b we tested whether the effect found in Experiment 2a was not just a priming effect of the first picture on the retrieval of the second picture name. Participants were presented with the same sequence of stimuli, but were now asked to name just the second object. This task is known as a picture-picture priming task. Several picturepicture priming experiments have been carried out before. In an experiment by Huttenlocher and Kubicek (1983) the effect of semantic relatedness in a picture-picture priming task was studied. The proportion of semantically related trials within a block was varried. Both the effect of relatedness and of proportion of related trials were found to be significant, as was the interaction effect. In the condition with a high proportion of related pairs, speech onset latencies are faster for related pairs. Sperber, McCauly, Ragain, and Weil (1979) studied the semantic picture-picture priming effect in combination with the effect of picture visibility. A significant interaction effect between visibility and relatedness was found, supporting the assumption that if visibility is low, more semantic information will be used to identify the pictures. The effect of relatedness within visibility conditions was in both cases facilitory. In an experiment by Glazer and Glazer (1989) two pictures were presented at different locations on the computer screen at different time points. The task of the participants was either to name the picture to appear first or to name the secondly presented picture. Only three related pairs were used. Semantic inhibition effects were found for almost all stimulus onset asynchrony conditions. Because the picture-picture priming effect was found to depend on the proportion of related trials (Huttenlocher & Kubicek, 1983) we decided to try to replicate the picturepicture priming effect for the materials and setup of Experiment 2a.

Results
Speech onset latencies were measured from the onset of the second picture. Errors like voice-key time-outs, incorrect triggers of the voice-key, disfluencies, incorrect word namings, incorrect determiners were removed from the data analysis. Only responses between 450 ms and 1600 ms were analyzed. Mean response times and error rates for each of the conditions are shown in Table 5. The task proved to be very difficult. On average participants named object pairs incorrectly in 22% of the trials. For both types of relatedness a separate analysis of variance was carried out. The effect of presentation duration was significant both in the semantic relatedness condition (F1 ´1 19µ 12 95 p 0 002 F2 ´1 19µ 56 37 p 0 001), and in the phonological relatedness condition (F1´1 19µ 7 21 p 0 015 F2´1 19µ 18 97 p 0 001). No evidence was found of phonological relatedness (both F’s 1). The facilitation effect of semantic relatedness, however, was significant (F1 ´1 19µ 8 57 p 0 009 F2 ´1 19µ 4 61 p 0 045). No significant interaction effect was be found (F1 ´1 19µ 1 04 n s F2 1). Observed errors were often semantic in nature. However, similar numbers of semantic errors were found in the semantically related and in the unrelated condition (630 errors across participants in the related condition versus 627 in the unrelated condition). A sample of the errors is shown in Table 6.

Discussion
In Experiment 2a a significant effect of semantic relatedness of the picture names was found. There was no effect of phonological relatedness. Observed errors suggest that semantic information of the first object presented is retrieved before retrieval of the second object name takes place. It is as if participants stopped retrieving the first object name after the semantic information had been retrieved. No effect of phonological relatedness is found on response times and

Method
Participants. Twenty-four participants took part in the experiment. Three participants were removed from the analysis because

8
Table 6 Target Aap (’monkey’) Appel (’apple’) Bijl (’axe’) Bus (’bus’) Fluit (’flute’) Geweer (’gun’) Geweer (’gun’) Kast (’cupboard’) Leeuw (’lion’) Mes (’knife’) Peer (’pear’) Raket (’missile’)

HERMENS, MEYER, AND LEVELT

Observed Kat (’cat’) Peer (’pear’) Beitel (’chisel’) Trein (’train’) Piano (’piano’) Fluit (’flute’) Kanon (’cannon’) Stoel (’chair’) Tijger (’tiger’) Kat (’cat’) Banaan (’banana’) Hark (’rake’)

A sample of errors observed in Experiment 2a. Relatedness to the target Sematically related to the target Sematically related to the target Sematically related to the target Sematically related to the target Sematically related to the target Shape similar to the target Sematically related to the target Sematically related to the target Sematically related to the target ? Sematically related to the target Similar in shape

Table 7 Mean response times (in ms) and error rates (in parenthesis) by participants observed in Experiment 2b. 150 ms presentation duration Semantic Phonological Related 722 (6%) 703 (8%) Unrelated 728 (6%) 705 (7%) 350 ms presentation duration Semantic Phonological Related 722 (7%) 718 (7%) Unrelated 743 (10%) 705 (7%)

To investigate whether naming both pictures results in different effects than naming just one picture, an analysis of variance was carried out using the data of Experiment 2a and 2b, using task as a between subject variable. The only significant interaction was that between task and presentation duration. In addition, significant effects of task within each relatedness/presentation duration condition were found.

Discussion
If only the second picture has to be named an effect of semantically relatedness is found. The size of the relatedness effect is not significantly different from when both pictures have to be named, since no interaction effect is found between naming task and relatedness. However, task and presentation duration do interact. No effect of presentation duration is found if only the second picture is named. This suggest that all processing necesarry for the semantic relatedness facilitiation effect is finished 150 ms after picture presentation since adding additional processing time does not influence response times. The first object does not seem to be ignored by the participants since its relatedness with the second object is affecting respones times. It is possible that participants decide to process the first picture because they know this will speed up response times. Another possibility is that the decision to process the first picture is not under control of the participants, and that the first picture is automatically processed.

of voice-key problems. One participant who had also participated in Experiment 2a was also removed from the analysis. Materials, Design, Apparatus, and Procedure. Materials, design, apparatus, and procedure were the same as in Experiment 2a. In the instruction participants were asked to name the second picture using utterances like ’de beer’ (’the bear’). The voice-key timeout interval was changed from 3000 ms to 2500 ms.

Results
Incorrect word namings were removed from the analysis. Response times shorter than 450 and longer than 1250 ms were also removed from the analysis. Mean response times and error rates are presented in Table 7. Only the effect of semantic relatedness was significant (F1 ´1 19µ 5 72 p 0 027 F2 ´1 19µ 5 80 p 0 026). For the effect of phonological relatedness both F-values were less than 1. The interaction effects were not significant (interaction of semantic relatedness and presentation duration: F1 ´1 19µ 1 50 n.s., F2 1, interaction of phonological relatedness and presentation duration: F1 ´1 19µ 2 54 n.s., F2 ´1 19µ 1 28 n.s). In contrast to in Experiment 2a of no effect of presentation duration of the first object was found (for the semantic relatedness condition: F1 1, F2 ´1 19µ 3 18 n.s., for the phonological relatedness condition: F1 1, F2 ´1 19µ 2 11 n.s).

Experiment 2c
In the final experiment the dependence of the picturepicture priming effect on linguistic processing is investigated. In Experiment 2c participants are asked to decide whether the second object is an existing object or not. For this task pictures of non-objects are used. These pictures look like pictures of existing objects, but are in fact not existing in the real world (see, Kroll & Potter, 1984). If a picture-picture priming effect is found for the decision task this suggests that the effect does not depend on the retrieval of the object name. An effect of semantic relatedness might occur since semantic

PRODUCTION OF MULTIPLE WORDS UTTERANCES

9

properties of the object might be needed to decide whether the object is existing or not. Semantically related objects often tend to be similar in shape too. To see whether the semantic facilitation effects found in Experiments 2a and 2b was not due to the shape similarity of related objects, participants were asked to rate the shape similarity of each of the object pairs.

Table 8 Mean response times (in ms) and error rates (in parenthesis) by participants observed in Experiment 2c. 150 ms presentation 350 ms presentation Related 538 (1%) 512 (4%) Unrelated 542 (4%) 526 (5%)

Method, Part 1
Participants. Twenty participants from the Max Planck Institute participant pool who had not participated in Experiments 2a and 2b took part in the experiment. Materials. The semantically related pairs from Experiments 1a through 2b were used, together with 20 pictures of nonobjects. The phonologically related and unrelated pairs were not used because in Experiments 2a and 2b no effects of phonologically relatedness were found. Adding the nonobject pictures resulted in twice the number of trials of Experiments 2a and 2b. By removing the pictures of the phonologically related and unrelated pictures the experiment would take approximately as long as the other experiments. Two additional practice non-objects pictures were added to the picture set. Design. The experiment consisted of four blocks each consisting of 10 related object pairs, 10 unrelated pairs, and 20 object/non-object pairs. Related (e.g., ’bear and monkey’) and control pairs (e.g., ’bear and chair’) were presented in different blocks. Each block was presented twice. Once the first picture was presented for 150 ms, and once for 350 ms. The order of the presentation duration of the first picture was counterbalanced across participants. The order of the trials within each block was pseudo-randomized. This means that at most 5 successive equal requested responses were permitted. In addition, two successive trials were not allowed to contain the same object. Each block was preceded by one or two practice trials (in the beginning of the experiment and after the change of presentation duration two practice trials were presented), and followed by a break that could be ended by the participant by pressing one of the buttons of the button box. Apparatus. The same equipment was used as in Experiments 2a and 2b. The voice-key was replaced with a button box. Procedure. On each trial, participants were first presented with a fixation point for 600 ms, followed by a blank screen for 200 ms. Then the first picture was shown for 150 or 350 ms, followed by a blank screen for 100 ms, after which the second picture was presented for 1600 ms. During the presentation of the second picture, the participant tried to decide as fast as possible (trying to make as few errors as possible) whether the

picture was an existing object. In the case of an object, the right button had to be pressed independent of the handedness of the participant. In the case of a non-object a left button was requested. Before the experiment the participants received a written instruction, in which the task was explained. In addition the participants were asked to study two picture booklets, one containing pictures of objects, one showing the non-objects. Method, Part 2. Following the decision task, the same 20 participants completed a questionnaire on the shape similarity of each of the object pairs. One by one the picture pairs were presented on the computer screen, together with their item number. The participant was asked to rate the shape similarity of the objects on a five-point scale. The rating was filled in in a table on a sheet of paper containing the item numbers. After the score was entered, the participant pressed a button on the button box to get the next pair on the screen. The order of presentation of the pairs was randomized across participants.

Results
All incorrect object decision responses were removed from the data set. Mean response times and error rates by participants are presented in Table 8. Only responses to objects were analyzed. The difference in response times for the semantically related and unrelated priming condition is found to be marginally significant in the participant analysis (F1 ´1 19µ 3 77 p 0 067), and significant in the item analysis (F2 ´1 18µ 6 07 p 0 024). No effect of presentation duration of the first picture is found (F1 ´1 19µ 0 20 n.s., F2 ´1 18µ 0 94, n.s.), neither was there an interaction effect of relatedness and presentation duration of the first picture (F1 ´1 19µ 0 05 n.s., F2 ´1 18µ 0 40 n.s.). For each of the object pairs the mean similarity rating was computed. The ratings of semantically related pairs are higher than those for unrelated pairs (related: 3.47 on average, unrelated: 2.01), and this difference is significant (t ´38µ 5 22, p 0 001). However, the shape similarity is not a good predictor for the response times in the three experiments. For the response times of Experiment 2a, in which both of the pictures had to be named, the correlation with the similarity rating equals  0 0257 (n.s.). The correlation of the ratings with the response times for Experiment 2b, in which the second picture had to be named, equals  0 754 (n.s.). Finally, the correlation of the ratings with the response

10

HERMENS, MEYER, AND LEVELT

times of Experiment 2c (object/non-object decision) equalled r  0 1931 (n.s.).

Discussion
The decision whether an object in an existing object can be primed by semantically related objects. The priming effect is not due to the shape similarity of the objects since no significant correlation is found between response times and similarity ratings. In addition, shape similarity cannot account for the response times found in a naming tasks. This suggests that semantic information of the first object influences the processing of the second object and not shape information.

the mental lexicon. In addition the retrieval process must be very carefully planned, since if words are retrieved in parallel there is a higher probability of producing speech errors.

References
Dell, G. (1986). A spreading-activation theory of retrieval in sentence production. Psychological Review, 93(3), 283–321. Glaser, W. & D¨ ngelhoff, F. (1984). The time course of u picture-word interference. Journal of Experimental Pyschology: Human Perception and Performance, 10, 640–654. Huttenlocher, J. & Kubicek, L. (1983). The source of relatedness effects on naming latency. Journal of Experimental Psychology: Learning, Memory, and Cognition, 9(3), 486–496. Kroll, J. & Potter, M. (1984). Recognizing words, pictures, and concepts: A comparison of lexical, object, and reality decisions. Journal of Verbal Learning and Verbal Behavior, 23(1), 39–66. Martin, N., Weisberg, R. & Saffran, E. (1995). Variables influencing the occurrence of naming errors: Implications for models of lexical retrieval. Journal of Memory and Language, 28(4), 462–485. Van der Meulen, F., Meyer, A. & Levelt, W. (2001). Eye movements during the production of nouns and pronouns. Memory & Cognition, 29(3), 512–521. Meyer, A. & Van der Meulen, F. (2001). Phonological priming effects on speech onset latencies and viewing times in object naming. Psychonomic Bulletin and Review, 7(2), 314–319. Meyer, A. & Schriefers, H. (1991). Phonological facilitation in picture-word interference experiments: Effects of stimulus onset asynchrony and types of interfering stimuli. Journal of Experimental Psychology: Learning, Memory and Cognition, 25, 907–922. Meyer, A., Sleiderlink, A. & Levelt, W. (1998). Viewing and naming objects: Eye movements during noun phrase production. Cognition, 66(2), 25–33. Sperber, R., McCauley, C., Ragain, R. & Weil, C. (1979). Semantic priming effects on picture and word processing. Memory and Cognition, 7(5), 339–345.

General Discussion
If participants are asked to name objects presented together on a computer screen in a single utterence, no consistent effects of relatedness of the object names are found. The pattern of viewing times and the lack of effects of relatedness on response times suggest that participants use a very sequential naming strategy in order to avoid errors during language production. Most of the errors made were semantic in nature. For example, if a pear had to be named, participants would use the name of a banana, and not of a chair. The semantic errors suggest that semantically related words strongly compete for selection during the semantical encoding phase. If the incorrect word is selected its phonological code is retrieved correctly. When participants are asked to name two pictures presented one after another effects of semantic relatedness are found. Also, when only the second picture has to be named or decided on an effect of semantical relatedness is found. These results suggest that semantic properties of observed objects are often retrieved, even if these properties are not needed for carrying out a task. The results of Experiment 2a suggest that participants try just to retrieve the semantic properties of the first object to name the object later on. It seems that retrieving more information early in the preparation of the utterence is of disadvantage in naming the second object to be presented and is therefore avoided as much as possible. In spite of the fact that semantically related pictures are also more similar in shape, no correlation is found between shape similarity ratings and response times. This suggests that all effects that are found are indeed semantic in nature. The results of the experiments suggest that in the production of speech people try to keep the encoding of words within an utterance as separate as possible. Errors in speech production therefore seem to be the result of unsuccessful separation of encoding processes. Our results show that language production is even a more impressive achievement of human speakers than was thought before. One reason for parallel retrieval was thought to be the incredible speed at which language is produced. The current experimental results show that speakers are not just concerned with the retrieval and the combination of material

Experimental materials
In Table 9 the names of the pictures used the experiments are shown.

PRODUCTION OF MULTIPLE WORDS UTTERANCES

11

Table 9 Names of pictures used in the experiments. Phonologically Related Unrelated bed - berg (bed - mountain) hand - berg (hand - mountain) hand - harp (hand - harp) bed - harp (bed - harp) beer - been (bear - leg) schaar - been (scissors - leg) schaar - schaats (scissors - beer - schaats (bear - skate) skate) bom - bot (bomb - bone) kam - bot (comb - bone) kam - kar (comb - cart) bom - kar (bomb - cart) boot - boom (boat - tree) hart - boom (heart - tree) hart - hark (heart - rake) boot - hark (boat - rake) hert - hek (deer - fence) kat - hek (cat - fence) kat - kan (cat - can) hert - kan (deer - can) kok - kop (cook - cup) kok - blad (cook - leaf) blik- blad (dustpan - leaf) blik - kop (dustpan - cup) pan - pak (pan - suit) ton - pak (barrel - suit) ton - tol (barrel - top) pan - tol (pan - top) pijp - pijl (pipe - arrow) step - pijl (scooter - arrow) step - ster (scooter - star) pijp - ster (pipe - star) schaal - schaap (bowl - wiel - schaap (wheel sheep) sheep) wiel - wieg (wheel - cradle) schaal - wieg (bowl - cradle) boek - boer (book - farmer) wolk - boer (cloud - farmer) wolk - worst (cloud - boek - worst (book sausage) sausage) Semantically Related Unrelated appel - citroen (apple - ezel - citroen (donkey lemon) lemon) ezel - varken (donkey - pig) appel - varken (apple - pig) arm - voet (arm - foot) spin - voet (spider - foot) spin - rups (spider - caterpil- arm - rups (arm - caterpillar) lar) kast - bureau (wardrobe - taart - bureau (pie - desk) desk) taart - brood (pie - bread) kast - brood (wardrobe bread) aap - leeuw (ape - lion) schip - leeuw (ship - lion) schip - kano (ship - canoe) aap - kano (ape - canoe) bus - trein (bus - train) rits - trein (zip - train) rits - knoop (zip - button) bus - knoop (bus - button) gitaar - fluit (guitar - flute) geweer - fluit (gun - fluit) geweer - kanon (gun - can- gitaar - kanon (guitar - cannon) non) zaag - beitel (saw - chisel) oog - beitel (eye - chisel) oog - mond (eye - mouth) zaag - mond (saw - mouth) raket - vliegtuig (missile - trompet - vliegtuig (trumpet aeroplane) - aeroplane) trompet - viool (trumpet - vi- raket - viool (missile - vioolin) lin) stoel - kruk (chair - stool) bijl - kruk (axe - stool) bijl - hamer (axe - hammer) stoel - hamer (chair - hammer) peer - banaan (pear - ba- deur - banaan (door - banana) nana) deur - raam (door - window) peer - raam (pear - window)