Unnoticed regularity violation elicits change-related brain activity

Unnoticed regularity violation elicits change-related brain activity

Biological Psychology 80 (2009) 339–347 Contents lists available at ScienceDirect Biological Psychology journal homepage: www.elsevier.com/locate/bi...

711KB Sizes 0 Downloads 0 Views

Biological Psychology 80 (2009) 339–347

Contents lists available at ScienceDirect

Biological Psychology journal homepage: www.elsevier.com/locate/biopsycho

Unnoticed regularity violation elicits change-related brain activity Istva´n Czigler a,b,*, Lı´via Pato´ a,b a b

Institute for Psychology of the Hungarian Academy of Sciences, P.O. Box 398, 1394 Budapest, Hungary University of Debrecen, Debrecen, Hungary

A R T I C L E I N F O

A B S T R A C T

Article history: Received 11 August 2008 Accepted 1 December 2008 Available online 7 December 2008

Event-related brain electric activity (ERP) was investigated to unnoticed visual changes. The orientation of grid elements (vertical or horizontal) changed after the presentation of 10–15 identical stimuli. The grid patterns were task irrelevant, and were presented in the background of a shape discrimination task. During the first half of the session, participants were unaware of the stimulus change. However, in comparison to the ERPs to the fifth identical stimuli, stimulus change elicited posterior negativities in the 270–375 ms range (visual mismatch negativity, vMMN). With participants instructed on the stimulus change, negativities emerged with earlier onset and with wider distribution. When stimulus change was preceded by only two identical stimuli, there were no such ERP effects. As the results show, a longer sequence of identical unattended stimuli may establish the memory representation of stimulus regularity, and violation of regularity is indicated by posterior negative ERP components (vMMN). ß 2008 Elsevier B.V. All rights reserved.

Keywords: Change-detection Event-related potential (ERP) Visual mismatch negativity Change blindness

1. Introduction Detection of environmental changes is one of the important tasks of the perceptual systems. However, visual changes frequently avoid conscious detection. The aim of the present study was to investigate the possibility of implicit registration of such changes using the method of event-related brain potentials (ERPs). In recent years, there has been increasing interest in an ERP component, closely related to implicit change-detection effects. The visual mismatch negativity (vMMN) is a homolog of the auditory mismatch negativity (MMN; for a review on the auditory studies, see Schro¨ger, 2007; Winkler, 2007). VMMN, a posterior negative component in the 120–400 ms range, is elicited by stimuli violating the regularity of environmental stimulation. In the majority of studies, vMMN is investigated in the oddball paradigm where the regularity is set up by a frequently presented stimulus (standard), and the regularity is violated by the presentation of an infrequent (deviant) stimulus (for a recent review, see Czigler, 2007). Usually standard and deviant are task-irrelevant back-

* Corresponding author at: Institute for Psychology of the Hungarian Academy of Sciences, P.O. Box 398, 1394 Budapest, Hungary. Tel.: +36 1 354 2290; fax: +36 1 354 2416. E-mail address: [email protected] (I. Czigler). 0301-0511/$ – see front matter ß 2008 Elsevier B.V. All rights reserved. doi:10.1016/j.biopsycho.2008.12.001

ground stimuli, or they are irrelevant features of task-related stimuli. VMMN is calculated as the difference between the ERPs elicited by stimuli violating the regularity and those corresponding to the regularity. VMMN was recorded for task-irrelevant deviations in color (e.g. Czigler et al., 2002; Kimura et al., 2006a,b), spatial frequency (Heslenfeld, 2003), stimulus contrast (Stagg et al., 2004), motion direction (Kremla´cˇek et al., 2006; PazoAlvarez et al., 2004), shape (Besle et al., 2005; Maekawa et al., 2005; Tales et al., 1999), line orientation (Astikainen et al., 2004; Kimura et al., 2008b), stimulus location (Berti and Schro¨ger, 2006), and facial expressions (Zhao and Li, 2006). In these studies, vMMN emerged as a posterior component. The precise localization of the origin of this component is uncertain. Czigler et al. (2004) reported retinotopic occipital origin for color-related vMMN, Pazo-Alvarez et al. (2004) obtained occipito-temporal distribution for motion deviancy. Recently, Yucel et al. (2007) conducted an fMRI study to assess the source of activity underlying the vMMN component. They recorded deviant-related BOLD signals from the V1, V2, right fusiform gyrus, and also from the posterior parietal cortex and from prefrontal regions. VMMN is also elicited by the violation of temporal regularities (Czigler et al., 2006a), and complex rules, like an irregular stimulus repletion instead of regular stimulus changes (Czigler et al., 2006b). In this respect, it is similar to the (auditory) MMN (Horva´th et al., 2001; Paavilainen et al., 2001, 2007). We suggested that vMMN is not only a correlate of activity difference to pre- and

340

I. Czigler, L. Pato´ / Biological Psychology 80 (2009) 339–347

post-change stimuli. The antecedent condition of vMMN emergence is the registration of environmental regularities.1 In order to consider vMMN as a homolog of the auditory MMN, it is important to demonstrate that the eliciting stimuli are not only task irrelevant, but these events are outside the field of focal attention. Unfortunately the majority of research on the field did not control this methodological aspect. In some studies the relevant object contained the deviant feature (e.g. Berti and Schro¨ger, 2006; Kimura et al., 2008b), or the irrelevant stimuli were presented in temporal contingency with the task stimuli within an otherwise empty field (e.g. Kimura et al., 2008a). In other studies task-relevant (target) and irrelevant (standard and deviant) stimuli were the members of the sequence, i.e. at a given moment only the irrelevant stimuli appeared on the screen (e.g. Amenedo et al., 2007; Stagg et al., 2004; Tales et al., 1999). It is difficult ‘‘not to attend’’ to a sole stimulus on an otherwise empty visual field. The problem is similar with studies using auditory tasks (Astikainen et al., 2004; Horimoto et al., 2002; Maekawa et al., 2005; Zhao and Li, 2006). Even in studies attempting to obtain information about the possibility of conscious detection of the task-irrelevant changes, such information was collected in rather informal ways (e.g. Winkler et al., 2005). In the present study, we attempted to investigate the effects of non-detected changes on ERP activity. The possibility of incidental detection of task-irrelevant stimulus change was investigated by using a semi-structured interview. In case of any hint about (real or false) detection of task-irrelevant stimulus changes ERP data were omitted. In some respects the vMMN paradigms are similar to the flicker methods investigating the change blindness phenomenon (for reviews see e.g. Simons & Levin, 1997; Simons and Rensink, 2005). In both paradigms blank periods between subsequent nonidentical stimuli prevents attentional capture by change-related local transients. Furthermore, in both cases changes became obvious after proper instructions or cues. However, the typical change blindness paradigm is a search task, whereas in the vMMN experiments the task is unrelated to the changing stimuli. In the change blindness studies changes remain unnoticed as long as no focal attention is directed to the location of stimulus change. In the vMMN task the experimenter assumes that no focal attention is directed to the changing stimuli throughout the whole stimulus sequence. In the majority of change blindness studies complex scenes were presented (but see e.g. Laloyaux et al., 2006; Simons, 2000; Thornton and Fernandez-Duque, 2000), whereas in vMMN experiments the stimuli were simple patterns (but see Zhao and Li, 2006). Despite such differences, vMMN studies may provide evidences about the possibility of implicit detection of visual changes in cases of non-reported stimulus changes. In this respect the results may support the view about the possibility of comparison of pre- and post-change scenes without the involvement of focal attention (Fernandez-Duque and Thornton, 2003; Hollingworth et al., 2001; Laloyaux et al., 2006; Thornton and Fernandez-Duque, 2000). Brain activity correlates of non-detected changes were investigated in some studies, but the results are inconclusive. In some studies, change-related event-related brain activity emerged only at detected changes (Henderson and Orbach, 2006; Eimer and 1 Posterior negativity is sometimes accompanied by a frontal positivity (Heslenfeld, 2003), and sometimes preceded by a posterior positive component (change-related positivity; Kimura et al., 2005, 2006a,b). Kimura et al. (2008b) considered this positivity as a correlate of the mismatch between the memory representation of frequent stimuli and the incoming stimulus, whereas these authors considered the negativity as a selective refractoriness effect (see also Kenemans, 2003; Mazza et al., 2005). For arguments favoring the memory dependence of the vMMN see Czigler (2007), Winkler et al. (2005), and Kimura et al. (2008b).

Mazza, 2005; Koivisto and Revonsuo, 20032; Niedeggen et al., 2004; Pourtois et al., 2006; Turatto et al., 2002). Contrary to such negative findings, Fernandez-Duque et al. (2003) obtained some hints about brain electric activity related to implicit change detection. Participants searched the display for stimulus change. As an ERP difference between undetected changing stimulation and stimulus repetition, these authors recorded an anterior positivity to changing sequences in the 240–300 ms latency range. In an fMRI study Beck et al. (2001) compared the activity to detected and undetected changes of secondary task stimuli. Undetected stimulus change elicited activity in posterior areas (fusiform and lingual gyri) and also in the inferior frontal gyrus. In Experiment 1, we applied a roving standard paradigm. In this paradigm, a series of identical stimuli are presented. After several repetitions the stimulation changes and the new stimuli are repeated several times. In a stimulus block, such sequences are presented repeatedly. VMMN-related stimuli were presented in the background of task-related stimuli. We compared the ERPs for the first (change) stimuli of a sequence (i.e., stimuli deviating from the regularity) to ERPs elicited by regular stimuli. At half-time of the session, the participants were interviewed about their observations on the background stimulation. Thereafter, we explained the characteristics of the background stimulation and repeated the procedure. In Experiment 2, short regular change/repetition sequences (AABBAABB. . .) were introduced, i.e., changes did not violate any regulations. In other respects, the two experiments were identical. 2. Experiment 1 2.1. Methods 2.1.1. Participants Participants were 20 paid students. They had normal color vision, and normal or corrected-to-normal visual acuity. A written consent was obtained from the participants. Due to the large number of artifacts, the data of three participants were discarded. According to the half-time interview, in three participants there were some hints that they noticed stimulus change in the first part of the session. Of the remaining 14 participants, there were 8 females and 6 males (mean age: 21.6 years, range: 19–23). 2.1.2. Apparatus and procedure The stimuli were green grid patterns on red background (for half of the participants) or red grid patterns on a green background (for the other half of the participants). The red and green colors had luminances of 25.4 and 24.2 cd/m2, respectively. Stimuli covered a field of 9.38 (vertical)  13.38 (horizontal) within a 17 in. screen (1280  1024 pixel; 85 Hz refreshing rate), except for a black stripe of 1.78 thickness at the middle of the screen (task field). The thickness of the lines of the grid pattern was 1 pixel. The size of the rectangle elements of the grid was 0.58 and 0.258. The orientation of the rectangles was either horizontal or vertical (see Fig. 1). The viewing distance was 120 cm. On the upper half-field of the screen, the orientation of the patterns was identical within the whole session (horizontal for half of the participants and vertical for the other half), whereas on the lower half-field the orientation of the rectangle elements of the grid pattern was organized into sequences of 10–15 stimuli of identical orientation (the orientation changed after the last stimulus of the sequence).3 The grid patterns appeared for 68 ms with 702-ms interstimulus interval (ISI). Within a stimulus block, there were 600 stimuli (48 sequences). In the center of the task field, a red (in case of green background) or green (in case of red background) quadrangle (target) was presented. The vertical size of the quadrangle was 0.58. From time to time, the width of the quadrangle alternated between 0.58 and 0.78 (i.e., from square to rectangle and vice versa). Such changes did not appear simultaneously with the onset of the grid patterns, and the target events were unrelated to the grid sequence. The interval between the changes of the target quadrangle was random, within 15–30 s. The task was to indicate the change of the quadrangle. The participants pressed a response button (‘‘as fast as possible’’) whenever they detected the change. In the session, four stimulus blocks were presented. After the second block, the participants answered questions about their experiences on the stimulation (see 2 Note that in this study the behavioral results indicated the implicit registration of stimulus change. 3 In a previous study vMMN appeared at the lower half-field stimulation (Czigler et al., 2004); see also Amenedo et al. (2007).

I. Czigler, L. Pato´ / Biological Psychology 80 (2009) 339–347

341

Fig. 1. Outline of the experimental design of Experiment 1.

Appendix A). In case of any hints about detection of any change (real or unreal) of the background stimulation, the session was terminated. Otherwise, the participants were instructed about the structure of the stimulus sequences, and two other stimulus blocks were presented with the same task. 2.1.3. Measurement of brain electric activity EEG was recorded (DC-200 Hz, sampling rate 1000 Hz, Synamp2 amplifiers, NeuroScan recording system with Ag/AgCl electrodes placed on 61 scalp locations (modified 10–20 international system) using an elastic electrode cap (EasyCap). The reference electrode was attached to the linked mastoids and the ground electrode to the forehead. The horizontal EOG was recorded with a bipolar configuration between electrodes positioned lateral to the outer canthi of the two eyes. The vertical EOG was recorded with a bipolar configuration between electrodes placed above and below the right eye. EEG signals were filtered offline (0.10–30 Hz, 24 dB) and epochs of 600 ms duration, starting 100 ms before the stimulus onset were averaged separately for the first (change-related) and fifth (a regular) stimuli of all subsequences. Two averages were calculated, the first one for the two blocks before the interview (first half of the session), and the second for the two blocks after the interview (second half of the session). Epochs with an amplitude change exceeding 70 mV on any channel were rejected from further analysis. To identify change-related event-related activity, ERPs to the fifth stimuli within a sequence were subtracted from the first stimuli of the sequences. The choice of the fifth stimulus was motivated by two factors. First, vMMN is considered as an EEG correlate of violated regularity. Therefore it was necessary to establish regularity. Four identical stimuli were considered as a proper regular sequence (see e.g. Cowan et al., 1993a,b for auditory data). Second, in the second part of the session the participants were aware of the design. Therefore expectation of stimulus change gradually increased at the later part of sequence of identical stimuli. Such expectation may introduce an uncontrolled variability into the data. We assumed that the fifth stimulus was a reasonable compromise. From the difference potentials group averages were calculated. Electrode locations with the possibility of changerelated differences were determined from such averages. In the first step, series of ttests were conducted on subsequent 5-ms epochs on the difference potentials at locations of apparent differences.4 We considered the periods of at least three 4 Assessment of significance was planned to assess according to the method developed by Smid et al. (1999), which is a variant of the calculation suggested by Guthrie and Buchwald (1991). We intended to compare the number of subsequent significant (p < 0.05) t-tests within a latency range where no change-related differences were expected ( 100 to +130 ms post-stimulus) with the significant subsequent t-tests within a range with the possibility of change-related activity (140– 380 ms). Only periods with larger number of significant differences within the latter period would have been considered in further significant analyses. However, there were no significant differences within the 100 to +130 ms epoch. Accordingly, epochs with subsequent significant t-tests were used in further analyses. Note that choosing the locations for such t-tests and selecting the locations for the ANOVAs are based on the inspection of the data set. This way there is a possibility to miss elusive differences. However, at this stage of research we intended to use conservative methods.

subsequent 5-ms epochs with significant t-tests for further analyses. In the next step, voltage maps were calculated for the average activities of the latency ranges showing differences on the basis of the significant t-tests. Sets of electrode clusters of at least three electrodes indicating change-related difference) were involved into further analyses. In such analyses, ANOVAs with factors of Stimulus (first vs. fifth stimuli within the subsequences) and Electrode locations were used. When appropriate, the Greenhouse–Geisser correction was used.

2.2. Results 2.2.1. Behavioral results From the 17 participants there were only three who noted that the grid pattern was not the same throughout the sequences. However, neither of these participants reported the exact nature of the change. Data of the three participants were discarded from further analyses. At the end of the second half of the session, the participants claimed that they were well aware of the changing stimulus background. Table 1 shows the RT, hit rate, and false alarm data. According to t-tests, there were no significant differences between the first and second part of the session.

2.2.2. Event-related potentials In the first part of the study (i.e., without the explanation of the changing grid pattern), we obtained two epochs revealing activity difference after the change and after the fifth identical stimulus. These epochs appeared between 270– 290 ms and 360–375 ms after the stimulus onset. As Fig. 2 shows, within these epochs the change-related activity was more negative than the activity elicited by a regular stimulus. In the earlier epoch, the change-related difference was confined to right posterior locations (P2, P4, P6, PO4, PO8). In an ANOVA with factors of Stimulus (change vs. regular) and Electrode location, the main effect of Stimulus was significant F(1, 13) = 5.66, h2 = 0.30, p < 0.05. The Electrode location main effect was also significant, F(5, 65) = 3.65, e = 0.37, h2 = 0.22, p < 0.05, showing that even within the right posterior areas there were amplitude differences in the change-related activity. In the 360–375 ms epoch, the changerelated difference was restricted to the P4, PO4, and PO8 locations. According to the ANOVA, in this epoch only, the Stimulus main effect was significant, F(1, 13) = 4.93, h2 = 0.27, p < 0.05.

Table 1 Reaction time, hit rate and false alarm number in the first and second part of the session (S.D. in parenthesis). Experiment 1.

First part Second part

Reaction time

Hit rate

False alarma

448.7 (102.4) 479.2 (117.4)

85.3 (7.21) 84.2 (7.20)

1.6 (2.37) 0.50 (0.85)

a Due to the small number of false alarm responses, we present the number of false alarms, instead of the false alarm rate.

342

I. Czigler, L. Pato´ / Biological Psychology 80 (2009) 339–347

Fig. 2. (A) Electrode locations with significant ERP differences (first minus fifth stimuli) in Experiment 1, first half. (B) Group average (n = 14) ERPs for the first and fifth stimuli of the sequences and the difference potential in Experiment 1, first half. Vertical bars indicate the epochs with change-related activity. (C) The average of the activities from six electrode locations with significant effect in the 270–290 ms range is shown. As Fig. 3 shows, in the second part change-related differences emerged with different latency and different scalp distribution. The earliest epoch with significant difference appeared in the 205–220 ms range. The broad change-related negativity involved anterior as well as posterior regions over the left hemisphere. According to an ANOVA with Stimulus and Electrode factors (F5, F7, FT7, FC3, FC5, T7, C3, C5, CP3, CP7, P3, P5, PO3, PO7, O1) the main effect of Stimulus was significant, F(1, 13) = 8.31, h2 = 0.39, p < 0.05. The Electrode main effect was also significant, F(14, 182) = 6.53, e = 0.12, h2 = 0.33, p < 0.01. In the 230–240 ms epoch, the negativity expanded to a large part of the scalp, as shown by the significant Stimulus main effect, F(1, 13) = 12.8, h2 = 0.50, p < 0.01. In this analysis, the whole set of electrodes was used

in the Electrode location factor, but we obtained no Location effect. There were no significant effects in an intermediate range, but Stimulus main effect became significant again in the 305–330 ms range, as a bilateral posterior negativity. In an ANOVA with Stimulus and Electrode factors (P1, P5, P7, Pz, P2, P3, P4, P6, P8, PO3, POz, PO4, PO7, PO8, O1, Oz, O2), both the Stimulus main effect [F(1, 13) = 7.77, h2 = 0.37, p < 0.05] and the Location main effect were significant [F(16, 208) = 3.58, e = 0.21, h2 = 0.22, p < 0.05]. The latest change-related difference emerged in the 380–440 ms range. On the left side, the activity to the change-stimulus elicited larger negativity than the regular stimulus. This effect has frontal (F1, F3, F5, F7, FT7) and parieto-temporal (T7, C5, CP5, P5, PO7) locations. In both the frontal and

I. Czigler, L. Pato´ / Biological Psychology 80 (2009) 339–347

343

Fig. 3. (A) Electrode locations with significant ERP differences (first minus fifth stimuli) in Experiment 1, second half. (B) Group average (n = 14) ERPs (in representative locations) for the first and fifth stimuli of the sequences and the difference potential in Experiment 1, second half. Vertical bars indicate the epochs with change-related activity. (C) The average of activities from the electrodes with significant effect in the 205–220 and 305–330 ms ranges. the parieto-temporal analyses, only the Stimulus main effect was significant, F(1, 13) = 5.35, h2 = 0.29, p < 0.05 and F(1, 13) = 5.55, h2 = 0.30, p < 0.05, respectively.5

5

We made an attempt to localize the change-related activities. Loreta reconstruction (Pascual-Marqui et al., 1994) localized the difference in the 270– 290 ms of the first part of the session into the cuneus and right superior temporal gyrus. The later (360–375 ms) effect was localized into the middle and superior temporal gyri. Concerning the second part, the earlier (205–220 ms) effect was localized into the cuneus, precuneus, left middle and superior temporal gyri, whereas the later effect (305–330 ms) was localized into the middle and superior temporal gyri, and into the anterior cingular cortex.

2.3. Discussion In the first half of the session, the vast majority of participants did not notice any change in the lower half of the background visual field, even if such changes were well above threshold. However, after 10–15 identical stimuli, such background changes elicited a negative-going shift over the posterior locations. This negativity lateralized to the right side, and it was significant in the 270–290 and 360–375 ms epochs. As the difference potentials show, this activity started after 200 ms, and it was a continuous

I. Czigler, L. Pato´ / Biological Psychology 80 (2009) 339–347

344

shift of 200 ms duration. The negative shift is similar in latency and location to the component recorded to deviant stimuli in some studies (Czigler et al., 2006b; Kimura et al., 2008b; Tales et al., 1999; Stagg et al., 2004; Zhao and Li, 2006). In other vMMN studies, deviant-related negativity emerged as a more phasic component with somewhat shorter latency (see Czigler, 2007, for a review). It is difficult to consider such negative shift as a refractory effect of any exogenous ERP component. The deviant-related ERP changes were markedly different in the second part of the session. The negativity emerged earlier as an activity over the left side, continued as a broad shift, and terminated as a posterior bilateral negativity. From such results, it seems that the instruction about the background characteristics created activity that was not only stronger, but also qualitatively different from the automatic registration of the background change. Before concluding that the negativity to undetected changes was due to the violation of a rule established by four identical stimuli, and not only a consequence of stimulus change, we conducted a control experiment with regular stimulus changes and stimulus repetitions. 3. Experiment 2 In this experiment, neither stimulus change nor stimulus repetition violated any regularity. On one hand, in this situation a system sensitive to stimulus change per se is expected to generate distinctive responses. No change-related activity is expected if the system is sensitive only in case of violation of an established regularity. 3.1. Methods 3.1.1. Participants A new sample of 12 participants (5 females and 7 males; mean age: 22.7 years, range: 19–28) from the same population was investigated. In this case, none of the participants mentioned any observation about the structure of the background stimulation. 3.1.2. Apparatus, procedure, and measurement of brain electric activity The stimuli were identical to those of Experiment 1. The versions of the grid pattern (horizontal and vertical) were presented in a regular AABBAABB. . . order. The task, number of stimuli within a block, the number of blocks, and the structure of the experimental session were identical in the two experiments. The same electrophysiological apparatus and the same data processing methods were used. ERPs were averaged separately for the change and repetition stimuli, and differences were calculated between the ERPs elicited by the two stimulus types.

3.2. Results and discussion 3.2.1. Behavioral results Table 2 shows the RT, hit rate, and false alarm data. Having received instructions about the regularity of background stimulation, all the subjects were able to detect the AABBAABB. . . pattern. According to t-tests, there was no significant RT difference between the first and second part of the session. However, hit rate was lower in the second part [t(11) = 2.37, p < 0.05)]. Attention devoted to the grid pattern in the second half may explain the performance decrement.

3.2.2. Event-related potentials ERPs for the change and repetition stimuli are shown in Fig. 4. Using the same method as in Experiment 1, we were unable to identify any difference between the repeated and changing stimuli in the first part of the session. Table 2 Reaction time, hit rate and false alarm number in the first and second part of the session (S.D. in parenthesis). Experiment 2.

First part Second part

Reaction time

Hit rate

False alarma

493.2 (119.7) 474.1 (137.2)

81.5 (15.5) 73.2 (17.3)

0.0 0.2

a Due to the small number of false alarm responses, we present the number of false alarms, instead of the false alarm rate.

In the second part of the session, the only significant effect appeared in the 350– 390 ms range at the occipito-parietal (P4, POz, PO4) locations. In this case, the change-stimulus elicited more negative response than the repetition stimulus. In an ANOVA with Stimulus and Location factors, Stimulus main effect was significant [F(1, 11) = 12.71, h2 = 0.54, p < 0.01]. It is difficult to relate this late effect to welldefined ERP components. At any rate, it is a hint of discrimination between the repeated and changing stimuli. However, in comparison to the wide distribution of the change-related negativities in the second half of Experiment 1, only a handful set of locations were involved in the change/repetition difference in Experiment 2.

4. General discussion The main result of the present study is an ERP difference between the regular stimuli of a roving standard sequences and the change stimuli of such sequences in case of undetected stimulus changes. The difference emerged as right posterior negativity in the 250–400 range (significant in the 270–290 and 360–375 ms epochs). The changes were well above threshold, because after receiving proper instruction all the participants detected the stimulus change. The negativities were elicited only if the eliciting stimuli violated a sequential regularity. As the results of Experiment 2 show, a regular change did not elicit similar ERP effect. In a previous study (Czigler et al., 2006b), we obtained similar results. In a regular (AABBAA. . .) sequence, changes did not elicit posterior negativity. In the Czigler et al. (2006b) study, an irregular repetition (violation of regularity) elicited posterior negativity with the latency comparable to the earlier negativity in the first part of the present study. In some studies the vMMN had similar latency values (e.g. Stagg et al., 2004; Tales et al., 1999; Zhao and Li, 2006). This latter statement is important for two reasons. First, in many studies vMMN appeared as a sharper negative wave. It seems that the latency and duration of negativities are dependent on the stimulus conditions. In case of less salient changes or in case of more complex rules (e.g. Czigler et al., 2006b) vMMN has longer latency and less phasic appearance. Second, the similarity between the present results and the results of studies with less stringent control of conscious change detection (e.g. Tales et al., 1999; Stagg et al., 2004; Zhao and Li, 2006) is an indication that vMMN in such studies did not depend on attentional processes. One may argue that in our roving standard procedure regularity involved the stimulus change, i.e., probability of change increased (from zero) after 10 identical stimuli, and it was 1.0 after 15 identical stimuli. However, implicit registration of such regularity is improbable. The minimum time between two stimulus changes was 7.7 s. While in the visual modality there are no compatible data, in the auditory modality regularities are detected only within a much shorter period of time (Sussman and Gumenyuk, 2005). An alternative interpretation of the present data might be that in Experiment 1 the repetitive stimulation elicited the refractory state of specific neuronal elements, whereas stimulus change stimulated a fresh population of neurons (Kimura et al., 2006b). Kimura et al. (2006b) obtained increasing posterior negativity to stimulus change as a function of the number of preceding repeated stimuli. While prediction of the refractoriness and the memory comparison mechanisms is similar, the data of the Kimura et al. (2006b) study are not decisive in the issue. Apart for some arguments favoring memory-based interpretation of the vMMN (Czigler, 2007, see also Kimura et al., 2008b), it is important to emphasize that instead of the modulation of a particular ERP component the negativity in the present study was a relatively long shift. In the range of the negativity, there were both positive and negative deflections, but the difference potential was a monophasic negativity. Furthermore, the negative difference potential(s) had longer onset latency than the exogenous components, i.e., the components with supposed refractory state. The selective refractoriness explanation faces difficulties in

I. Czigler, L. Pato´ / Biological Psychology 80 (2009) 339–347

345

Fig. 4. Group average (n = 14) ERPs at representative electrode locations for the first and fifth stimuli of the sequences and the difference potentials in Experiment 2, first and second half.

interpreting activity differences between the ERPs of two parts of the session. In the second half of Experiment 1, the change-related activity difference started earlier, and initially it lateralized to the left side as a widespread (anterior and posterior) activity. In the later latency ranges it expanded to the right hemisphere; in the 305– 330 ms range it was a bilateral posterior activity. As a most cautious notion, it is unlikely that the change-related effects of the second half of the session were only stronger versions of the effects that emerged in the first part. On the basis of the present results, it is premature to speculate on the location of the neural substrate involved in the registration of undetected changes. The earlier

posterior effect (270–290 ms) may involve prestriate structures. Recently, on the basis of LORETA reconstruction the precuneus was supposed to be involved in a wide range of visual cognitive activities (Cavanna and Trimble, 2006). Evidence for the activity of anterior structures appeared only in the second part of the session. Involvement of anterior structures in conscious detection and in attentional processes is well demonstrated. While opinions on neural correlates of consciousness are markedly different, many of them converge at this point (see, e.g. Beck et al., 2001; Dehane and Naccache, 2001; Lamme, 2006). It should be emphasized that in Experiment 2 we obtained no change-related ERP effect in the first part of the study, and in the

I. Czigler, L. Pato´ / Biological Psychology 80 (2009) 339–347

346

second part there was only a very late effect. As the divergent ERP results of the two experiments show, the system sensitive to the differences between consecutive visual stimuli is not only a simple change-detection mechanism. We suggest that to detect changes, such system has to build up a representation on the regularities of visual stimulation, and its activity is elicited by the violation of regularities. The oddball and roving standard procedures are particularly appropriate in creating representations about regularities, and we suggest that in these paradigms vMMN is an electrophysiological correlate of changes violating such regularities. However, as some behavioral (Fernandez-Duque and Thornton, 2003; Laloyaux et al., 2006), ERP (e.g. Fu et al., 2003; Kimura et al., 2005, 2006a), and fMRI (Beck et al., 2001) results show, task-irrelevant visual changes per se may have detectable influences on visual processing and brain activity. In Section 1 we emphasized that there are obvious differences between the change blindness and the MMN paradigms. Most importantly, the former is an active search task, whereas in latter the vMMN-related stimuli (or stimulus features) are irrelevant. However, at least in one respect the results of the vMMN paradigm may be relevant in connection to the change blindness effect. Change blindness phenomenon appears if no proper memory representation is available for the pre- and post-change stimuli (O’Regan and Noe¨, 2001), or there is no comparison between the representations (Simons et al., 2002). As vMMN results show, at least in case of regular stimulation there is a possibility for the development of memory representation, and the system is sensitive to the violation of regularity (i.e., there is a mismatch between a memory representation and the representation of the ongoing stimulation. The duration of the representation underlying vMMN is an open issue. In the majority of investigations the stimulus onset asynchrony (SOA) was shorter than 1 s, and in the study investigated temporal aspect of regularity (Czigler et al., 2006a), the regular pattern disappeared within a second. Concerning the capacity of the memory system, it seems that the memory system is capable of storing more than one item. This is because Winkler et al. (2005) reported vMMN in a study with two standards (frequent conjunctions of color and direction) and two deviants (rare conjunctions of color and direction). It is improbable that the memory underlying vMMN is closely related to the iconic memory (in its classical sense; see Sperling, 1960). Unlike the ‘‘icon,’’ vMMN resists visual masking (Czigler et al., 2007). Concerning other types of visual memory, Landman et al. (2003) described a large capacity store with longer (1.5 s) lifetime. On the basis of its capacity and duration, this memory may operate in case of implicit detection of violated regularities. The most important and open issue, however, is the functional role of the memory supporting a ‘‘primitive intelligence’’ (Na¨a¨ta¨nen et al., 2001), disclosed by the vMMN studies. Acknowledgement This research was supported by the National Science Foundation of Hungary (OTKA) 71600. Appendix A The sequence of questions in the half-time interview. 1. 2. 3. 4.

How much square-changes did you notice? Have you any feeling that you have missed square changes? Please estimate the duration of the experiment! What did you notice at the upper and lower half of the screen (above and below the dark line)? 5. Were the patterns identical at the upper and lower half-field?

6. 7. 8. 9. 10.

Were the patterns at the upper half-field always identical? In case of NO response: What kind of changes did you notice? Were the patterns at the lower half-field always identical? In case of NO response: What kind of changes did you notice? Did you notice any other change either at the upper or lower half-field?

Note that questions 1–3 are irrelevant for the purpose of the interview. References Amenedo, E., Pazo-Alvarez, P., Cadaveira, F., 2007. Vertical asymmetries in preattentive detection of changes in motion direction. International Journal of Psychophysiology 64, 184–189. Astikainen, P., Ruusuvirta, T., Wikgen, J., Korjonen, T., 2004. The human brain processes visual changes that are not cued by attended auditory stimulation. Neuroscience Letters 368, 231–234. Beck, D.M., Rees, G., Frith, C.D., Lavie, N., 2001. Neural correlates of change detection and change blindness. Nature Neuroscience 4, 645–650. Berti, S., Schro¨ger, E., 2006. Visual distraction: a behavioral and event-related brain potential study in humans. NeuroReport 17, 151–155. Besle, J., Fort, A., Giard, M.H., 2005. Is the auditory sensory memory sensitive to visual information? Experimental Brain Research 166, 337–344. Cavanna, A.E., Trimble, M.R., 2006. The precuneus: a review of its functional anatomy and behavioural correlates. Brain 129, 564–583. Cowan, N., Winkler, I., Teder, W., Na¨a¨ta¨nen, R., 1993a. Memory prerequistes of mismatch negativity in the auditory event-related potential (ERP). Journal of Experimental Psychology: Learning, Memory and Cognition 19, 909–921. Cowan, N., Winkler, I., Teder, W., Na¨a¨ta¨nen, R., 1993b. Short- and long-term prerequisites of the mismatch negativity in the auditory event-related potential (ERP). Journal of Experimental Psychology: Learning, Memory, and Cognition 19, 909–921. Czigler, I., 2007. Visual mismatch negativity: violation of non-attended environmental regulations. Journal of Psychophysiology 21, 224–230. Czigler, I., Bala´zs, L., Winkler, I., 2002. Memory-based detection of task-irrelevant visual change. Psychophysiology 39, 869–873. Czigler, I., Bala´zs, L., Pato´, L., 2004. Visual change detection: event-related potentials are dependent on stimulus location in humans. Neuroscience Letters 364, 149– 153. Czigler, I., Winkler, I., Pato´, L., Va´rnagy, A., Weisz, J., Bala´zs, L., 2006a. Visual temporal window of integration as revealed by the visual mismatch negativity event-related potential to stimulus omission. Brain Research 1104, 129–140. Czigler, I., Weisz, J., Winkler, I., 2006b. ERPs and deviance detection: visual mismatch negativity to repeated visual stimuli. Neuroscience Letters 401, 178–182. Czigler, I., Weisz, J., Winkler, I., 2007. Backward masking and visual mismatch negativity: electrophysiological evidence for memory-based detection of deviant stimuli. Psychophysiology 44, 610–619. Dehane, S., Naccache, L., 2001. Towards a cognitive neuroscience of consciousness: basic evidence and a workspace framework. Cognition 79, 1–7. Eimer, M., Mazza, V., 2005. Electrophysiological correlates of change detection. Psychophysiology 42, 328–342. Fernandez-Duque, D., Thornton, I.M., 2003. Explicit mechanisms do not account for implicit localization and identification of change: an empirical replay to Mitroff et al. (2000). Journal of Experimental Psychology: Human Perception and Performance 29, 846–858. Fernandez-Duque, D., Grossi, G., Thornton, I.M., Neville, H.J., 2003. Representation of change: separate electrophysiological markers of attention, awareness and implicit processing. Journal of Cognitive Neuroscience 15, 491–507. Fu, S., Fan, S., Chen, L., 2003. Event-related potentials reveal involuntary processing of orientation changes in visual modality. Psychophysiology 40, 770–775. Guthrie, D., Buchwald, J.S., 1991. Significance testing of difference potentials. Psychophysiology 28, 240–244. Henderson, R.M., Orbach, H.S., 2006. Is there a mismatch negativity during change blindness? NeuroReport 17, 1011–1015. Heslenfeld, D.J., 2003. Visual mismatch negativity. In: Polich, J. (Ed.), Detection of Change: Event-related Potential and fMRI Findings. Kluver Academic Press, Boston, pp. 41–59. Hollingworth, A., Williams, C.C., Henderson, J.M., 2001. To see and remember: visually specific information is retained in memory from previously attended objects in natural scenes. Psychological Bulletin and Review 8, 761–768. Horimoto, R., Inagaki, M., Yano, T., Sata, Y., Kaga, M., 2002. Mismatch negativity of the color modality during a selective attention task to auditory stimuli in children with mental retardation. Brain and Development 24, 703–709. Horva´th, J., Czigler, I., Sussman, E., Winkler, I., 2001. Simultaneously active preattentive representations of local and global rules for sound sequences. Cognitive Brain Research 12, 131–144. Kimura, M., Katayama, J., Murohashi, H., 2005. Positive difference in ERPs reflect independent processing of visual changes. Psychophysiology 42, 369–379. Kimura, M., Katayama, J., Murohashi, H., 2006a. Independent processing of visual stimulus changes in ventral and dorsal stream features indexed by an early positive difference in event-related brain potentials. International Journal of Psychophysiology 59, 141–150.

I. Czigler, L. Pato´ / Biological Psychology 80 (2009) 339–347 Kimura, M., Katayama, J., Murohashi, H., 2006b. Probability-independent and dependent ERPs reflecting visual change detection. Psychophysiology 43, 180– 218. Kimura, M., Katayama, J., Murohashi, H., 2008a. Attention switching function of memory-comparison-based change detection system in the visual modality. International Journal of Psychophysiology 67, 101–113. Kimura, M., Katayama, J., Hira, H., Schro¨ger, E., 2008b. Visual mismatch negativity: new evidence from the equiprobable paradigm. Psychophysiology 45 (Suppl. 1), S123. Kremla´cˇek, J., Kubova´, Z., Kuba, M., Langrova´, J., 2006. Visual mismatch negativity elicited by the magnocellular system activation. Vision Research 46, 485–490. Kenemans, J.L., 2003. Detection of visual change: mismatch or rareness? Neuroreport 14, 1239–1242. Koivisto, M., Revonsuo, A., 2003. An ERP study of change detection, change blindness, and visual awareness. Psychophysiology 40, 423–429. Laloyaux, C., Destrebecqz, A., Cleeremans, A., 2006. Implicit change identification: a replication of Fernandez-Duque and Thornton (2003). Journal of Experimental Psychology: Human Perception and Performance 32, 1366–1379. Lamme, V.A.F., 2006. Towards a true neural stance on consciousness. Trends in Cognitive Sciences 10, 494–501. Landman, R., Spekreijse, H., Lamme, V.A.F., 2003. Large capacity storage of integrated objects before change blindness. Vision Research 43, 149–164. Maekawa, T., Goto, Y., Kinukawa, N., Taniwaki, T., Hanbu, S., Tobimatsu, S., 2005. Functional characterization of mismatch negativity to visual stimulus. Clinical Neurophysiology 116, 2392–2402. Mazza, V., Turatto, M., Sarlo, M., 2005. Rare stimuli or rare changes: What really matters for the brain. Neuroreport 16, 1061–1064. Na¨a¨ta¨nen, R., Tervaniemi, M., Sussman, E., Paavilainen, P., Winkler, I., 2001. Primitive intelligence in the auditory cortex. Trends in Neurosciences 24, 283–288. Niedeggen, M., Heil, M., Ludowig, E., Rolke, B., Harris, C.L., 2004. Priming illusory words: and ERP approach. Neuropsychologia 47, 745–753. O’Regan, J.K., Noe¨, A., 2001. A sensorimotor account of vision and visual consciousness. Behavioral and Brain Sciences 24, 939–1031. Paavilainen, P., Simola, J., Jaramillo, M., Na¨a¨ta¨nen, R., Winkler, I., 2001. Preattentive extraction of abstract feature conjunctions from auditory stimulation as reflected by mismatch negativity (MMN). Psychophysiology 38, 359–365. Paavilainen, P., Araja¨rvi, P., Takegata, R., 2007. Preattentive detection of nonsalient contingencies between auditory features. NeuroReport 18, 159–163. Pascual-Marqui, R.D., Michel, C.M., Lehmann, D., 1994. Low resolution electromagnetic tomography: a new method for localizing electrical activity in the brain. International Journal of Psychophysiology 18, 49–65.

347

Pazo-Alvarez, P., Amenedo, E., Cadaveira, F., 2004. Automatic detection of motion direction change in the human brain. European Journal of Neurosciences 19, 1978–1986. Pourtois, G., De Prett, M., Hauert, C.-A., Vuilleumier, P., 2006. Time course of brain activity during change blindness and change awareness: performance is predicted by neural events before change onset. Journal of Cognitive Neuroscience 18, 2108–2129. Schro¨ger, E., 2007. Mismatch negativity: a microphone into auditory memory. Journal of Psychophysiology 21, 138–146. Simons, D.J., 2000. Current approaches to change blindness. Visual Cognition 7, 1– 15. Simons, D.J., Levin, D.T., 1997. Change blindness. Trends in Cognitive Sciences 1, 261–267. Simons, D.J., Rensink, R.A., 2005. Change blindness: past, present and future. Trends in Cognitive Sciences 9, 16–20. Simons, D.J., Chabris, C.F., Schnur, T., 2002. Evidence for preserved representation in change blindness. Consciousness and Cognition 11, 78–97. Smid, H.G.O.M., Jakob, A., Heinze, H.J., 1999. An event-related brain potential study of visual selective attention to conjunctions of color and shape. Psychophysiology 36, 264–279. Sperling, G., 1960. The information available in brief visual presentation. Psychological Monograph 74 Whole No. 498. Stagg, C., Hindley, P., Tales, A., Butler, S., 2004. Visual mismatch negativity: the detection of stimulus change. NeuroReport 15, 659–663. Sussman, E.S., Gumenyuk, V., 2005. Organization of sequential sounds in auditory memory. NeuroReport 13, 1519–1523. Tales, A., Newton, P., Troscianko, T., Butler, S., 1999. Mismatch negativity in the visual modality. NeuroReport 10, 3363–3367. Thornton, I.M., Fernandez-Duque, D., 2000. An implicit measure of undetected change. Spatial Vision 14, 21–44. Turatto, M., Angrilli, A., Mazza, V., 2002. Looking without seeing the background change: electrophysiological correlates of change detection versus change blindness. Cognition 84, B1–B10. Winkler, I., 2007. Interpreting the mismatch negativity. Journal of Psychophysiology 21, 147–163. Winkler, I., Czigler, I., Sussman, E., Horva´th, J., Bala´zs, L., 2005. Preattentive binding of auditory and visual stimuli. Journal of Cognitive Neuroscience 17, 320– 339. Yucel, G., McCarthy, G., Belger, A., 2007. fMRI reveals that involuntary visual deviance processing is resource limited. NeuroImage 34, 1245–1252. Zhao, L., Li, J., 2006. Visual mismatch negativity elicited by facial expressions under non-attentional condition. Neuroscience Letters 410, 126–131.