Discussion
The present study examined a series of lateralized ERP components and the lateralized alpha-band suppression at parieto-occipital electrodes to elucidate the mechanisms involved in the representation of grouped features in VWM. The behavioral results revealed a large grouping benefit for detecting orientation changes; additionally, there was a smaller, but reliable benefit for color changes – thus directly replicating our previous behavioral study (Chen et al., 2021a). Analyses of the ERPs revealed the early PPC to already mirror the gradual variation of behavioral change-detection performance for grouped, partially grouped, and ungrouped configurations. Moreover, there was a significant negative relationship between behavioral accuracy and the PPC in the grouped and partially grouped configurations, indicative of individual variations of grouping upon memory performance being already reflected in these early lateralizations. The PPC effect might be associated with preattentive object integration (e.g., Nikolaev et al., 2008; Kasai et al., 2015). Of note, however, grouping was also found to influence subsequent attentional and memory processing stages, as evidenced by the modulations of the N1pc, N2pc and CDA waves, which were driven in particular by the fully grouped star object. In addition, the lateralized alpha amplitude was gradually modulated by the presented object configurations, with most suppression occurring with the ungrouped, followed by partially grouped, and least suppression with grouped configurations. Taken together, these findings indicate that object integration at early perceptual stages influences processing at subsequent stages, thereby facilitating focal-attentional processing and the subsequent maintenance of individual objects and their constituent features, thus leading to enhanced precision of grouped representations stored in VWM.
Our results confirm that perceptual grouping provides an efficient means to combine multiple elements into higher-order units, consistent with previous reports of improved memory performance when multiple features can be represented as a coherent (grouped) object, as compared to the same set of features distributed across multiple, separate items (e.g., Chen et al., 2021a; Fougnie et al., 2013; Luria & Vogel, 2011; Olson & Jiang, 2002; Xu, 2002). In this view, grouping may enhance memory by providing an integrated, higher-order (superordinate) object representation (e.g. a ”star” or a ”triangle”), with this global object facilitating the comparison with the probe and thereby freeing VWM resources. This may explain why not only the grouping-relevant feature (orientation) but also the grouping-irrelevant feature (color) benefits from the superordinate object representation (at least when attention is set to process whole objects, i.e., when both color and orientation are task-relevant, see Chen et al., 2021a). For instance, the improved VWM representation of orientation features with grouped configurations may free memory resources that are then available to also process the color features of the to-be-memorized objects in greater detail.
Going beyond previous findings, our results indicate that memorizing features (i.e., orientation and color) in grouped objects involves multiple, sequential stages of processing that can be traced using lateralized ERPs (see also Kasai et al., 2015). Interestingly, this benefit of grouping in VWM already arises at an early perceptual level of processing (as evidenced by the modulations in the PPC component). In this regard, our study is the first to show that effects of grouping upon (higher-order) processing of visual information in VWM already originate at an early perceptual level of processing (i.e., in the PPC). In earlier studies (e.g., Nikolaev et al., 2008; see also Nikolaev et al., 2016; Esposito et al., 2023), the P1 amplitudes were found to be affected by, and negatively correlated with, grouping sensitivity in a (perceptual) discrimination task. This is consistent with the present results that also showed negative correlations between the PPC amplitudes and the (grouping-related) orientation detection performance. For instance, the larger the PPC amplitude, the lower the orientation detection accuracies, suggesting that an increase in early visual processing demands (as indicated by a larger PPC), coincides with comparably reduced performance for remembering the orientation features. Consistent with this result, the PPC revealed the largest positivity for ungrouped, followed by partially grouped and least for grouped configurations, suggesting that the extraction of visual information is more demanding for ungrouped items relative to “more” grouped objects.
Preferential processing of a coherent grouped object (as compared to ungrouped fragments) was also evident in the N1pc (see also Martinez et al., 2007; Murray et al., 2002; Senkowski et al., 2005). This effect was mainly driven by the large and reliable difference between grouped and partially grouped/ungrouped configurations, while revealing no significant difference between partially grouped and ungrouped configurations. An obvious reason for the non-significant difference between ungrouped and partially grouped configurations could relate to overall task difficulty. For instance, our observers were required to memorize six different colors and six different orientations - which is clearly above the usual maximum VWM capacity estimate of three to four items (Luck & Vogel, 1997). ERP differences between the partially grouped and ungrouped configurations might thus be difficult to resolve given that performance clearly operates beyond the usual capacity limits. In the grouped condition, performance improved quite substantially (revealing an orientation detection accuracy of 77% as compared to smaller differences for the partially grouped (65%) and ungrouped (60%) configurations). Due to this variation in the size of the behavioral effect, the N1pc results pattern could thus mirror the overall efficiency in selecting and representing the partially grouped and ungrouped configurations (the same might also be true for the subsequent N2pc and CDA effects, where the difference between partially grouped and ungrouped configurations was also non-significant). However, beyond these difficulties to resolve the gradual increase in grouping strength in some of the ERPs, the “overall” grouping modulations in PPC and N1pc together nevertheless clearly show that integrated objects modulate the bottom-up attentional deployment towards to-be-memorized items at early processing stages.
Following these, variations of grouping strength were found to also modulate the N2pc component, which – in the current study - likely reflects the engagement of focal attention (Eimer, 1996) by the to-be-memorized item. The current study yielded a larger N2pc for grouped relative to ungrouped and partially grouped configurations, which likely reflects a more pronounced focus of (focal) attention towards more regular, grouped objects (see also Senkowski et al., 2005; Conci et al., 2006; 2011; Töllner et al., 2015). In previous visual search studies that employed Kanizsa-type configurations (e.g. Conci et al., 2006), the target was always defined as a Kanizsa figure while the distractors (in the opposite hemifields) varied in terms of grouping strength (alongside with a variation in target-distractor similarity). In these cases, stronger grouping in distractors would result in a broader attentional tuning, thus reducing the (focal) attentional engagement towards the Kanizsa target figure. In the current change-detection task, grouping strength was always manipulated in the task-relevant memory array, which would conversely result in a larger N2pc for the grouped configuration relative to the less grouped configurations. These findings thus coincide in that grouping in distractors diverts attentions away from the target (see Conci et al., 2006), while enhanced grouping in the target configuration leads to an enhanced attentional focus (see Conci et al., 2011). Grouping thus appears to influence both early perceptual processes and the subsequent attentional selection and engagement, with attention being facilitated by the processing of the grouped items (e.g., Marini & Marzi, 2016; Rauschenberger &Yantis, 2001; Senkowski et al., 2005; Wiegand et al., 2015).
Finally, the subsequent memory-maintenance stage exhibited a continuous grouping benefit, as revealed by an enhanced CDA for grouped, as compared to ungrouped and partially grouped, configurations. At first glance, this result seems inconsistent with previous evidence that suggested that perceptual grouping effectively reduces the effective number of “items” in the display, thereby leading to reduced CDA amplitudes (Gao et al., 2011; Peterson et al., 2015). Of note, however, these previous studies usually manipulated grouping by means of item similarity, which is different from the current study, where disparate objects were bound into a unified global object by grouping on the basis of closure (and collinearity). For example, it was reported in previous studies that similar colors were compressed in VWM such that the CDA amplitude for these colors was reduced and essentially comparable in amplitude to just one to-be-memorized color (Gao et al., 2011; Peterson et al., 2015). In contrast, the six different colors and the six orientations used in the present study were maximally different from each other and they could therefore not be represented in a compressed format, which might then be seen in the CDA. Moreover, in the current study, the physical stimulation was always identical while only the grouping strength differed across conditions. The current study therefore provides evidence for an unconfounded influence of grouping upon VWM, while controlling other factors relating to the stimulus itself.
A major characteristic of the CDA is that its amplitude increases with the number of objects maintained in VWM (for a review, see Luria et al., 2016), while usually reaching an asymptote at about 3-4 items, which reflects the maximum capacity (Cowan, 2001; Luck & Vogel, 2013). The current study presented 6 items (with 6 distinct orientations and6 colors) in all conditions, which is clearly above the usual capacity limit. Interestingly, our results showed that grouping can lead to a substantial capacity enhancement up to an estimate of around 5.5 items with grouped configurations – relative to 3.8 and 3.0 items for partially grouped and ungrouped configurations, respectively. This increase in the overall K estimates was also mirrored in CDA variations, which suggests that the grouping-dependent increase in memory capacity is reflected in the CDA measure.
The CDA appears to not only represent the passive storage of individual items, but also reflects the active representation of globally completed objects in VWM, which may in turn require additional mnemonic resources. For instance, a sustained increase in the CDA amplitude (and a concurrent improvement in performance) was found to be associated with objects that are completed despite partial occlusion - as compared to physically identical, yet uncompleted object fragments (Chen et al., 2018b). It is thus possible that “modal” completion of Kanizsa figures exhibits similar VWM storage properties and reveals comparable CDA modulations than objects that are integrated on the basis of “amodal” completion (e.g., due to partial occlusion), given that modal and amodal completions are associated with comparable behavioral patterns (e.g., Chen et al., 2018a) and partially overlapping neural mechanisms (Murray et al., 2004).
A complementary pattern of results was additionally revealed from the analysis of the oscillatory amplitudes. For instance, we found a graded, grouping-strength-related modulation in the lateralized alpha suppression during the maintenance phase, which was comparable to the observed PPC variation during initial perceptual processing. Specifically, the observed differences in the lateralized alpha amplitudes that we observed were associated mainly with contralateral variations, that is, they reflect the processing of the task-relevant stimulus configurations rather than the inhibition of task-irrelevant placeholders (see also Bacigalupo & Luck 2019; Thut et al., 2006; Noonan et al., 2016). Moreover, our findings show that the lateralized alpha suppression increased as grouping strength decreased, thus possibly reflecting the ‘effort’ in representing individual features of less vs. more strongly grouped objects despite a constant number of to-be-encoded locations. Posterior-occipital alpha has been suggested to reflect top-down adjustments of attentional control (e.g., Thut et al., 2006; Murphy et al. 2020; Wang et al., 2019; 2021; Woodman et al., 2022). In agreement with this interpretation, the variation of lateralized alpha in the current study may index the larger attentional demands required to process the less (vs. more) grouped configurations. This may also explain the similarity of the result pattern revealed for the lateralized alpha suppression to concurrent variations in the PPC component since both neuronal signatures are associated with comparable (attentional) processing requirements (albeit at different levels of processing).
We would thus propose that the CDA and lateralized alpha suppression might be mapped onto two separable cognitive mechanisms of VWM, relating to (i) the representation of individual objects and (ii) associated attentional control processes, respectively (Unsworth et al. 2014; see also Fukuda et al., 2015). That is, the CDA effect as a function of grouping might be more consistent with mechanisms necessary for holding multiple individuated representations in an active state over the duration of the retention interval. And the increased lateralized alpha suppression for more ungrouped elements might in turn be directly associated with the increased attentional control demands for keeping the individual representations accessible during maintenance when the displayed stimulus configuration increases in perceptual complexity. In fact, we found that when quantifying grouping by means of the difference between grouped and ungrouped configurations, the CDA and the lateralized alpha amplitude were marginally correlated (r = 0.30,p = 0.075). This indicates that the two neurophysiological signals are likely manifestations of linked neural processes (see also van Dijk et al., 2010; Mazaheri & Jensen, 2008; but see Fukuda et al., 2015), with lower effort and higher precision in representing features of the grouped configuration as compared to higher effort and lower precision relating to ungrouped configurations. While being overall consistent with our current findings, admittedly, this interpretation of the relation between the CDA and alpha activity is, to a certain extent, still speculative and it might thus be necessary to consolidate these interpretations in future studies.
Our current findings may also be related to a theoretical framework that we proposed previously (Chen et al., 2021a), and according to which objects might be stored in VWM in terms of a hierarchical structure, comprising basic feature-level representations and associated higher-order, object-level representations (see also Brady et al., 2011; Nie et al., 2017). The representation of information at different levels in this hierarchy would jointly determine the capacity and quality of VWM representations. Moreover, depending on the current task, only to-be-remembered objects and associated features would be encoded and/or represented (Bocincova & Johnson, 2019; Chen et al., 2021a; Serences et al., 2009; Woodman & Vogel, 2008). In this view, the encoding of grouping-relevant (orientation) features would give rise to the encoding of the superordinate grouped object (which would in turn be represented at a higher level in the hierarchical memory structure, and which is reflected by the effects seen in the early PPC). The superordinate object representation in turn strengthens the representation of the grouping-relevant features (via reciprocal-feedforward/feedback connections). Moreover, the superordinate object representation also enhances the attentional selection (as reflected by the enhanced N1pc and N2pc) of both grouping-relevant and -irrelevant features at the basic level of representation via a feedback connection as the complete object is brought into the “focus of attention” (see, e.g., Oberauer & Hein, 2012; Souza & Oberauer, 2017; Printzlau et al. 2022). In the subsequent retention stage, the number and precision of feature representations is improved by the (superordinate) grouped object representation which might be reflected by variations in the CDA amplitude, while concurrent alpha activity would index the attentional requirements to process these maintained objects (see Machizawa et al., 2012; Wang et al., 2019; 2021; Woodman et al., 2022).