Post your comments on classes and readings for 9/11 and 9/13 here.
11 thoughts on “Measuring phonological knowledge with ERPs”
Josh
Connolly and Phillips (1994) discuss the phoneme by semantic match/mismatch interaction in latency of the N400 (Table 2b); however, they do not discuss what appears to be a phoneme by semantic match/mismatch interaction in latency of the PMN (Table 1b). This interaction appears such that the latency of the PMN increases in the semantic mismatch conditions compared to the semantic match conditions and that this difference is greater in the phoneme mismatch conditions than in the phoneme match conditions.
The authors later state that the stimuli were not sufficiently controlled to detect a PMN after the initial onset of the terminal word (particularly in the phoneme match-semantic mismatch condition), but that the PMN presumably did occur. Doesn’t this effect of semantic match/mismatch on PMN latency suggest that a later PMN did occur in these semantic mismatch conditions, which would have increased the mean recorded latency of the PMN?
In discussing the PMN findings (i.e., being elicited by phoneme mismatch, but not semantic mismatch), Connolly and colleagues suggest that as a sentence is being processed, the context of the sentence facilitates a prediction of the terminal word. I am curious if there is any research on how quickly does the terminal word prediction form? Specifically, how much contextual information is needed to be processed in a sentence for a mismatch terminal word to elicit the PMN? And whether the ease or speed of the contextual based prediction for a terminal word varies based on possible factors such as language expertise (e.g., comparing monolingual to multilingual participants), or sentence processing expertise (e.g., comparing throughout stages of development beginning at an age of either pre- or early sentence reading skills)?
How “finished†does the sentence have to be to elicit either the N400 or PMN effects? Thus, are the processing responses going to be similar when the sentence is complete and the terminal word is matching/mismatching compared to a sentence which is incomplete? I’m thinking about times when reading a poorly photocopied paper and the endings of sentences are cut off, but it is still possible to understand what the sentence was about. Is this type of disruption or violation of expectation processed similarly to the violations tested in Connolly and colleagues (1994)?
Breen, Kingston & Sanders (2013) report that behaviorally, listeners hear illegal onset syllables as a legal category (e.g., hearing /dla/ as /gla/), whereas this is not the case for absent-but-legal onset syllables (e.g., hearing /bwa/ as /bwa/). This finding is discussed as evidence for why English speakers are able to produce absent-but-legal syllables. I wonder whether the absent-but-legal syllables used in this study (/bwa/ and /pwa/) are truly representative of being “absentâ€. As the authors point out, English speakers can produce and hear these syllables in Spanish loan words. It seems that words such as these, from a language in which /bwa/ and /pwa/ are legal syllables, have been somewhat incorporated into English in the sense that English speakers have experience hearing and producing these onset syllables. That is, do English speakers hear these syllables because the syllables are defined as “legal†in English or because speakers have experience with them? If other onset syllables, defined as illegal (e.g., /tl/), were borrowed more frequently in English, would English speakers report hearing this syllable?
Hillary brought up a good point about the distinction between illegal, absent, and present onset clusters in English. I think the question of how to label onsets such as /bw/ and /pw/ is really open. They are probably absent for at least some speakers of English, but certainly aren’t for others. Perhaps they should be considered “rare” rather than “absent.” However, I do think the difference in how listeners process /bw,pw/ and /dl,tl/ is an important one that suggests there is a difference between phonotactically illegal strings and ones that are not actually prohibited. To the extent that listeners actually hear /dl/ and /gl/, it wouldn’t matter how many times they heard it – the sound would still be categorized as /gl/. On the other hand, since the ERPs indicate that there are differences in how listeners are representing /dl/ and /gl/, there is hope that listeners could learn to access those veridical representations and start using them. Whether that would occur with loan words or only in individuals trying to learn a new language is hard to say.
Charisse’s raised questions about how much context is necessary to see a PMN and how dependent the effect is on being at the end of the sentence. As a short answer, this study was the first to report the PMN and so went with the approach of having the strongest manipulation possible – extremely high close probability ending sentences. However, presumably the PMN would be there as long as a listeners had some expectation. For example, a single word could be enough context if all the pairs in a study were opposites (e.g., right, wrong and good, bad) and then the participant heard the word “white.” The end of the sentence turns out not to be the best place to look for any type of ERP effect because of overlapping, low-frequency ERPs that tend to occur at the ends of phrases including sentences. If you can set up the expectations for the middle of a sentence, you can actually get cleaner effects.
With regards to Charisse’s question about terminal words, and Lisa’s response, does that mean that longer sentences necessarily provide noisier data? Since we don’t always speak in short sentences, are we, as researchers, resigned to a certain lack of ecological validity?
To push this terminal word/middle of the sentence question, one example sentence Connolly and Phillips provided was, “The dog chased the cat up the queen”. However, they mentioned they had sentences as short as 5 words, so I thought that “cat” might be interpreted as a terminal word for the sentence. If you had 2 mismatches (at the middle AND the terminal), of some controlled combination of phonemic/semantic mismatch, would that alter the effect of the ERPs of the latter word? In other words, if there is a mismatch at the middle of the sentence, do we get a similar mismatch ERP at the end of the sentence? A diminished effect? Or do participants sort of throw their hands up after one unexpected word and decide they have no idea what to expect after?
Although the MMN is assumed to be indexing cognitive processes, I was wondering if it is known how much refractory effects might play a role. Näätänen et al. (1997) appears to show a rather gigantic effect (Fig. 2), and I’m wondering how much is due to refractory effects within the populations of neurons processing the standard stimuli that are not present in the populations of neurons processing the deviant stimuli (in addition to how much of this effect is due to trial number differences between the standard and deviant). Furthermore, is it known whether refractory effects might play a role in categorical perception? Could it be the case that we’d see a release from refractory effects when subjects cross a perceptual boundary between one category and another?
@Tina In terms of expectations, longer sentences do not necessarily yield noisier data; it depends upon whether the context restricts or expands the space of plausible continuations.
For instance, compared to the context “The man removed the cake from the…(oven)”, context (1) below would increase the Cloze probability of ‘oven’ sentence finally, because even though the context is longer, the added context restricts the space of plausible continuations to things that can be both warm and glowing. On the other hand, context (2) below would not increase the Cloze probability of ‘oven’ sentence finally because the set of items from which a cake can be removed and that can also belong to a neighbor’s aunt is not more restrictive than the set of all items from which a cake can be removed.
(1) “The man removed the cake from of the warm, glowing…(oven)”
(2) “The man removed the cake from the neighbor’s aunt’s…(oven).”
Thanks, Josh, for pointing out that it is the extent to which a sentence provides a highly constraining context that matters most. Adrian Staub also has some data that suggests the cloze probability of words other than the highest cloze probability word matters as well. That is, a sentence that is completed with the same word 60% of the time and 1 of 40 other words each 1% of the time is processed differently than a sentence that is completed with the same word 60% of the time and 1 other word 40% of the time.
My comment about the end of the sentence had more to do with ERPs than with how constraining the sentences are. Sentence position, length, and degree of constraint can all vary independently. However, there is an ERP effect, specifically a low-frequency positive going wave that tends to occur at the end of sentences. This effect has been called the “Closure Positive Shift.” There is some evidence that the amplitude of the CPS is related to how large the phrase boundary is (larger at the end of a topic than a sentence, and larger at the end of a sentence than a phrase within a sentence) as well as how certain a listener is that the phrase is over. Some have argued that getting an unexpected word makes people think the phrase or sentence isn’t really over yet. By having the critical word occur at the end of the sentence, at best your PMN and/or N400 effects will rest on top of a large positive going wave. At worst, the negative ERP differences for sounds and words that don’t match expectations at the end of a sentence could simply be a reduction in the Closure Positive Shift because the unexpected words indicate to the listener that the sentence isn’t really over yet.
Using more than one violation in a sentence has been used as a way to get at whether specific ERP effects are automatic or controlled. The argument is that once the first violation is encountered, the listener will give up on trying to make sense of the sentence so any subsequent responses to a later violation must be automatic. The results are somewhat mixed. If you include two lexical violations of the type Connelly and Phillips use in the same sentence, the N400 is response to the second violation is greatly reduced but not absent. No one has done the PMN manipulation with 2 violations. When 2 syntactic violations were included in the same sentence, some researchers observe an intact left-anterior negativity (LAN) but a reduced P600 to the 2nd violation and argue that the LAN must reflect automatic processing. However, others show more of an N400 like effect to the 2nd syntactic violation.
IN RE: Ben
The relationship between the MMN and refractory effects (smaller auditory evoked potentials in response to sounds that were recently preceded by identical or similar sounds) is hard to pull apart. For most studies, you could argue that the physical differences between the deviants and the standards result in a release from refractory effects compared to the repetition of identical standards. This idea would also fit with the fact that people typically observe a larger MMN for more physically different standards and deviants. However, there are two arguments against the idea that the MMN is entirely a refractory effect. The first is simply that the MMN overlaps with both a negative going peak (N1) and a positive going peak (P2). Refractory effects alone should result in a larger N1 for deviants, consistent with the MMN, but also a larger P2 for deviants, with the opposite polarity of the MMN. The second is that you can get an MMN based on patterns of sounds and even to an unexpected silence. With these more complex patterns, refractory periods can’t explain the effects.
The question of categorical perception and refractory effects is an interesting one, but only to the extent that you accept the idea that language experience can shape early acoustic processing. That is, refractory effects are thought to reflect the “dumb” processes of populations of auditory neurons responding vigorously to the first presentation of a sound and then having a period of time in which the neurotransmitters and other molecules are less available for those same neurons to build up a response again. Categorical refractory effects would require that individual neurons respond to specific speech sounds (e.g., that you have neurons which respond to /b/ but not /p/ in a categorical manner). If instead, the representation of speech sounds is coded by the pattern of firing across multiple neurons, the differences in the response to /b/ and /p/ might be the timing or order in which the exact same neurons build up postsynaptic potentials. If that is the case, then the refractory effects themselves would not be categorical.
Connolly and Phillips (1994) discuss the phoneme by semantic match/mismatch interaction in latency of the N400 (Table 2b); however, they do not discuss what appears to be a phoneme by semantic match/mismatch interaction in latency of the PMN (Table 1b). This interaction appears such that the latency of the PMN increases in the semantic mismatch conditions compared to the semantic match conditions and that this difference is greater in the phoneme mismatch conditions than in the phoneme match conditions.
The authors later state that the stimuli were not sufficiently controlled to detect a PMN after the initial onset of the terminal word (particularly in the phoneme match-semantic mismatch condition), but that the PMN presumably did occur. Doesn’t this effect of semantic match/mismatch on PMN latency suggest that a later PMN did occur in these semantic mismatch conditions, which would have increased the mean recorded latency of the PMN?
In discussing the PMN findings (i.e., being elicited by phoneme mismatch, but not semantic mismatch), Connolly and colleagues suggest that as a sentence is being processed, the context of the sentence facilitates a prediction of the terminal word. I am curious if there is any research on how quickly does the terminal word prediction form? Specifically, how much contextual information is needed to be processed in a sentence for a mismatch terminal word to elicit the PMN? And whether the ease or speed of the contextual based prediction for a terminal word varies based on possible factors such as language expertise (e.g., comparing monolingual to multilingual participants), or sentence processing expertise (e.g., comparing throughout stages of development beginning at an age of either pre- or early sentence reading skills)?
How “finished†does the sentence have to be to elicit either the N400 or PMN effects? Thus, are the processing responses going to be similar when the sentence is complete and the terminal word is matching/mismatching compared to a sentence which is incomplete? I’m thinking about times when reading a poorly photocopied paper and the endings of sentences are cut off, but it is still possible to understand what the sentence was about. Is this type of disruption or violation of expectation processed similarly to the violations tested in Connolly and colleagues (1994)?
Breen, Kingston & Sanders (2013) report that behaviorally, listeners hear illegal onset syllables as a legal category (e.g., hearing /dla/ as /gla/), whereas this is not the case for absent-but-legal onset syllables (e.g., hearing /bwa/ as /bwa/). This finding is discussed as evidence for why English speakers are able to produce absent-but-legal syllables. I wonder whether the absent-but-legal syllables used in this study (/bwa/ and /pwa/) are truly representative of being “absentâ€. As the authors point out, English speakers can produce and hear these syllables in Spanish loan words. It seems that words such as these, from a language in which /bwa/ and /pwa/ are legal syllables, have been somewhat incorporated into English in the sense that English speakers have experience hearing and producing these onset syllables. That is, do English speakers hear these syllables because the syllables are defined as “legal†in English or because speakers have experience with them? If other onset syllables, defined as illegal (e.g., /tl/), were borrowed more frequently in English, would English speakers report hearing this syllable?
Hillary brought up a good point about the distinction between illegal, absent, and present onset clusters in English. I think the question of how to label onsets such as /bw/ and /pw/ is really open. They are probably absent for at least some speakers of English, but certainly aren’t for others. Perhaps they should be considered “rare” rather than “absent.” However, I do think the difference in how listeners process /bw,pw/ and /dl,tl/ is an important one that suggests there is a difference between phonotactically illegal strings and ones that are not actually prohibited. To the extent that listeners actually hear /dl/ and /gl/, it wouldn’t matter how many times they heard it – the sound would still be categorized as /gl/. On the other hand, since the ERPs indicate that there are differences in how listeners are representing /dl/ and /gl/, there is hope that listeners could learn to access those veridical representations and start using them. Whether that would occur with loan words or only in individuals trying to learn a new language is hard to say.
Charisse’s raised questions about how much context is necessary to see a PMN and how dependent the effect is on being at the end of the sentence. As a short answer, this study was the first to report the PMN and so went with the approach of having the strongest manipulation possible – extremely high close probability ending sentences. However, presumably the PMN would be there as long as a listeners had some expectation. For example, a single word could be enough context if all the pairs in a study were opposites (e.g., right, wrong and good, bad) and then the participant heard the word “white.” The end of the sentence turns out not to be the best place to look for any type of ERP effect because of overlapping, low-frequency ERPs that tend to occur at the ends of phrases including sentences. If you can set up the expectations for the middle of a sentence, you can actually get cleaner effects.
With regards to Charisse’s question about terminal words, and Lisa’s response, does that mean that longer sentences necessarily provide noisier data? Since we don’t always speak in short sentences, are we, as researchers, resigned to a certain lack of ecological validity?
To push this terminal word/middle of the sentence question, one example sentence Connolly and Phillips provided was, “The dog chased the cat up the queen”. However, they mentioned they had sentences as short as 5 words, so I thought that “cat” might be interpreted as a terminal word for the sentence. If you had 2 mismatches (at the middle AND the terminal), of some controlled combination of phonemic/semantic mismatch, would that alter the effect of the ERPs of the latter word? In other words, if there is a mismatch at the middle of the sentence, do we get a similar mismatch ERP at the end of the sentence? A diminished effect? Or do participants sort of throw their hands up after one unexpected word and decide they have no idea what to expect after?
Although the MMN is assumed to be indexing cognitive processes, I was wondering if it is known how much refractory effects might play a role. Näätänen et al. (1997) appears to show a rather gigantic effect (Fig. 2), and I’m wondering how much is due to refractory effects within the populations of neurons processing the standard stimuli that are not present in the populations of neurons processing the deviant stimuli (in addition to how much of this effect is due to trial number differences between the standard and deviant). Furthermore, is it known whether refractory effects might play a role in categorical perception? Could it be the case that we’d see a release from refractory effects when subjects cross a perceptual boundary between one category and another?
@Tina In terms of expectations, longer sentences do not necessarily yield noisier data; it depends upon whether the context restricts or expands the space of plausible continuations.
For instance, compared to the context “The man removed the cake from the…(oven)”, context (1) below would increase the Cloze probability of ‘oven’ sentence finally, because even though the context is longer, the added context restricts the space of plausible continuations to things that can be both warm and glowing. On the other hand, context (2) below would not increase the Cloze probability of ‘oven’ sentence finally because the set of items from which a cake can be removed and that can also belong to a neighbor’s aunt is not more restrictive than the set of all items from which a cake can be removed.
(1) “The man removed the cake from of the warm, glowing…(oven)”
(2) “The man removed the cake from the neighbor’s aunt’s…(oven).”
IN RE: Tina and Josh
Thanks, Josh, for pointing out that it is the extent to which a sentence provides a highly constraining context that matters most. Adrian Staub also has some data that suggests the cloze probability of words other than the highest cloze probability word matters as well. That is, a sentence that is completed with the same word 60% of the time and 1 of 40 other words each 1% of the time is processed differently than a sentence that is completed with the same word 60% of the time and 1 other word 40% of the time.
My comment about the end of the sentence had more to do with ERPs than with how constraining the sentences are. Sentence position, length, and degree of constraint can all vary independently. However, there is an ERP effect, specifically a low-frequency positive going wave that tends to occur at the end of sentences. This effect has been called the “Closure Positive Shift.” There is some evidence that the amplitude of the CPS is related to how large the phrase boundary is (larger at the end of a topic than a sentence, and larger at the end of a sentence than a phrase within a sentence) as well as how certain a listener is that the phrase is over. Some have argued that getting an unexpected word makes people think the phrase or sentence isn’t really over yet. By having the critical word occur at the end of the sentence, at best your PMN and/or N400 effects will rest on top of a large positive going wave. At worst, the negative ERP differences for sounds and words that don’t match expectations at the end of a sentence could simply be a reduction in the Closure Positive Shift because the unexpected words indicate to the listener that the sentence isn’t really over yet.
Using more than one violation in a sentence has been used as a way to get at whether specific ERP effects are automatic or controlled. The argument is that once the first violation is encountered, the listener will give up on trying to make sense of the sentence so any subsequent responses to a later violation must be automatic. The results are somewhat mixed. If you include two lexical violations of the type Connelly and Phillips use in the same sentence, the N400 is response to the second violation is greatly reduced but not absent. No one has done the PMN manipulation with 2 violations. When 2 syntactic violations were included in the same sentence, some researchers observe an intact left-anterior negativity (LAN) but a reduced P600 to the 2nd violation and argue that the LAN must reflect automatic processing. However, others show more of an N400 like effect to the 2nd syntactic violation.
IN RE: Ben
The relationship between the MMN and refractory effects (smaller auditory evoked potentials in response to sounds that were recently preceded by identical or similar sounds) is hard to pull apart. For most studies, you could argue that the physical differences between the deviants and the standards result in a release from refractory effects compared to the repetition of identical standards. This idea would also fit with the fact that people typically observe a larger MMN for more physically different standards and deviants. However, there are two arguments against the idea that the MMN is entirely a refractory effect. The first is simply that the MMN overlaps with both a negative going peak (N1) and a positive going peak (P2). Refractory effects alone should result in a larger N1 for deviants, consistent with the MMN, but also a larger P2 for deviants, with the opposite polarity of the MMN. The second is that you can get an MMN based on patterns of sounds and even to an unexpected silence. With these more complex patterns, refractory periods can’t explain the effects.
The question of categorical perception and refractory effects is an interesting one, but only to the extent that you accept the idea that language experience can shape early acoustic processing. That is, refractory effects are thought to reflect the “dumb” processes of populations of auditory neurons responding vigorously to the first presentation of a sound and then having a period of time in which the neurotransmitters and other molecules are less available for those same neurons to build up a response again. Categorical refractory effects would require that individual neurons respond to specific speech sounds (e.g., that you have neurons which respond to /b/ but not /p/ in a categorical manner). If instead, the representation of speech sounds is coded by the pattern of firing across multiple neurons, the differences in the response to /b/ and /p/ might be the timing or order in which the exact same neurons build up postsynaptic potentials. If that is the case, then the refractory effects themselves would not be categorical.