Discussion
The present study examined a series of lateralized ERP components and
the lateralized alpha-band suppression at parieto-occipital electrodes
to elucidate the mechanisms involved in the representation of grouped
features in VWM. The behavioral results revealed a large grouping
benefit for detecting orientation changes; additionally, there was a
smaller, but reliable benefit for color changes – thus directly
replicating our previous behavioral study (Chen et al., 2021a). Analyses
of the ERPs revealed the early PPC to already mirror the gradual
variation of behavioral change-detection performance for grouped,
partially grouped, and ungrouped configurations. Moreover, there was a
significant negative relationship between behavioral accuracy and the
PPC in the grouped and partially grouped configurations, indicative of
individual variations of grouping upon memory performance being already
reflected in these early lateralizations. The PPC effect might be
associated with preattentive object integration (e.g., Nikolaev et al.,
2008; Kasai et al., 2015). Of note, however, grouping was also found to
influence subsequent attentional and memory processing stages, as
evidenced by the modulations of the N1pc, N2pc and CDA waves, which were
driven in particular by the fully grouped star object. In addition, the
lateralized alpha amplitude was gradually modulated by the presented
object configurations, with most suppression occurring with the
ungrouped, followed by partially grouped, and least suppression with
grouped configurations. Taken together, these findings indicate that
object integration at early perceptual stages influences processing at
subsequent stages, thereby facilitating focal-attentional processing and
the subsequent maintenance of individual objects and their constituent
features, thus leading to enhanced precision of grouped representations
stored in VWM.
Our results confirm that perceptual grouping provides an efficient means
to combine multiple elements into higher-order units, consistent with
previous reports of improved memory performance when multiple features
can be represented as a coherent (grouped) object, as compared to the
same set of features distributed across multiple, separate items (e.g.,
Chen et al., 2021a; Fougnie et al., 2013; Luria & Vogel, 2011; Olson &
Jiang, 2002; Xu, 2002). In this view, grouping may enhance memory by
providing an integrated, higher-order (superordinate) object
representation (e.g. a ”star” or a ”triangle”), with this global object
facilitating the comparison with the probe and thereby freeing VWM
resources. This may explain why not only the grouping-relevant feature
(orientation) but also the grouping-irrelevant feature (color) benefits
from the superordinate object representation (at least when attention is
set to process whole objects, i.e., when both color and orientation are
task-relevant, see Chen et al., 2021a). For instance, the improved VWM
representation of orientation features with grouped configurations may
free memory resources that are then available to also process the color
features of the to-be-memorized objects in greater detail.
Going beyond previous findings, our results indicate that memorizing
features (i.e., orientation and color) in grouped objects involves
multiple, sequential stages of processing that can be traced using
lateralized ERPs (see also Kasai et al., 2015). Interestingly, this
benefit of grouping in VWM already arises at an early perceptual level
of processing (as evidenced by the modulations in the PPC component). In
this regard, our study is the first to show that effects of grouping
upon (higher-order) processing of visual information in VWM already
originate at an early perceptual level of processing (i.e., in the PPC).
In earlier studies (e.g., Nikolaev et al., 2008; see also Nikolaev et
al., 2016; Esposito et al., 2023), the P1 amplitudes were found to be
affected by, and negatively correlated with, grouping sensitivity in a
(perceptual) discrimination task. This is consistent with the present
results that also showed negative correlations between the PPC
amplitudes and the (grouping-related) orientation detection performance.
For instance, the larger the PPC amplitude, the lower the orientation
detection accuracies, suggesting that an increase in early visual
processing demands (as indicated by a larger PPC), coincides with
comparably reduced performance for remembering the orientation features.
Consistent with this result, the PPC revealed the largest positivity for
ungrouped, followed by partially grouped and least for grouped
configurations, suggesting that the extraction of visual information is
more demanding for ungrouped items relative to “more” grouped objects.
Preferential processing of a coherent grouped object (as compared to
ungrouped fragments) was also evident in the N1pc (see also Martinez et
al., 2007; Murray et al., 2002; Senkowski et al., 2005). This effect was
mainly driven by the large and reliable difference between grouped and
partially grouped/ungrouped configurations, while revealing no
significant difference between partially grouped and ungrouped
configurations. An obvious reason for the non-significant difference
between ungrouped and partially grouped configurations could relate to
overall task difficulty. For instance, our observers were required to
memorize six different colors and six different orientations - which is
clearly above the usual maximum VWM capacity estimate of three to four
items (Luck & Vogel, 1997). ERP differences between the partially
grouped and ungrouped configurations might thus be difficult to resolve
given that performance clearly operates beyond the usual capacity
limits. In the grouped condition, performance improved quite
substantially (revealing an orientation detection accuracy of 77% as
compared to smaller differences for the partially grouped (65%) and
ungrouped (60%) configurations). Due to this variation in the size of
the behavioral effect, the N1pc results pattern could thus mirror the
overall efficiency in selecting and representing the partially grouped
and ungrouped configurations (the same might also be true for the
subsequent N2pc and CDA effects, where the difference between partially
grouped and ungrouped configurations was also non-significant). However,
beyond these difficulties to resolve the gradual increase in grouping
strength in some of the ERPs, the “overall” grouping modulations in
PPC and N1pc together nevertheless clearly show that integrated objects
modulate the bottom-up attentional deployment towards to-be-memorized
items at early processing stages.
Following these, variations of grouping strength were found to also
modulate the N2pc component, which – in the current study - likely
reflects the engagement of focal attention (Eimer, 1996) by the
to-be-memorized item. The current study yielded a larger N2pc for
grouped relative to ungrouped and partially grouped configurations,
which likely reflects a more pronounced focus of (focal) attention
towards more regular, grouped objects (see also Senkowski et al., 2005;
Conci et al., 2006; 2011; Töllner et al., 2015). In previous visual
search studies that employed Kanizsa-type configurations (e.g. Conci et
al., 2006), the target was always defined as a Kanizsa figure while the
distractors (in the opposite hemifields) varied in terms of grouping
strength (alongside with a variation in target-distractor similarity).
In these cases, stronger grouping in distractors would result in a
broader attentional tuning, thus reducing the (focal) attentional
engagement towards the Kanizsa target figure. In the current
change-detection task, grouping strength was always manipulated in the
task-relevant memory array, which would conversely result in a larger
N2pc for the grouped configuration relative to the less grouped
configurations. These findings thus coincide in that grouping in
distractors diverts attentions away from the target (see Conci et al.,
2006), while enhanced grouping in the target configuration leads to an
enhanced attentional focus (see Conci et al., 2011). Grouping thus
appears to influence both early perceptual processes and the subsequent
attentional selection and engagement, with attention being facilitated
by the processing of the grouped items (e.g., Marini & Marzi, 2016;
Rauschenberger &Yantis, 2001; Senkowski et al., 2005; Wiegand et al.,
2015).
Finally, the subsequent memory-maintenance stage exhibited a continuous
grouping benefit, as revealed by an enhanced CDA for grouped, as
compared to ungrouped and partially grouped, configurations. At first
glance, this result seems inconsistent with previous evidence that
suggested that perceptual grouping effectively reduces the effective
number of “items” in the display, thereby leading to reduced CDA
amplitudes (Gao et al.,
2011; Peterson et al.,
2015). Of note,
however, these previous studies usually manipulated grouping by means of
item similarity, which is different from the current study, where
disparate objects were bound into a unified global object by grouping on
the basis of closure (and collinearity). For example, it was reported in
previous studies that similar colors were compressed in VWM such that
the CDA amplitude for these colors was reduced and essentially
comparable in amplitude to just one to-be-memorized color (Gao et al.,
2011; Peterson et al., 2015). In contrast, the six different colors and
the six orientations used in the present study were maximally different
from each other and they could therefore not be represented in a
compressed format, which might then be seen in the CDA. Moreover, in the
current study, the physical stimulation was always identical while only
the grouping strength differed across conditions. The current study
therefore provides evidence for an unconfounded influence of grouping
upon VWM, while controlling other factors relating to the stimulus
itself.
A major characteristic of the CDA is that its amplitude increases with
the number of objects maintained in VWM (for a review, see Luria et al.,
2016), while usually reaching an asymptote at about 3-4 items, which
reflects the maximum capacity (Cowan, 2001; Luck & Vogel, 2013). The
current study presented 6 items (with 6 distinct orientations and6 colors) in all conditions, which is clearly above the usual capacity
limit. Interestingly, our results showed that grouping can lead to a
substantial capacity enhancement up to an estimate of around 5.5 items
with grouped configurations – relative to 3.8 and 3.0 items for
partially grouped and ungrouped configurations, respectively. This
increase in the overall K estimates was also mirrored in CDA variations,
which suggests that the grouping-dependent increase in memory capacity
is reflected in the CDA measure.
The CDA appears to not only represent the passive storage of individual
items, but also reflects the active representation of globally completed
objects in VWM, which may in turn require additional mnemonic resources.
For instance, a sustained increase in the CDA amplitude (and a
concurrent improvement in performance) was found to be associated with
objects that are completed despite partial occlusion - as compared to
physically identical, yet uncompleted object fragments (Chen et al.,
2018b). It is thus possible that “modal” completion of Kanizsa figures
exhibits similar VWM storage properties and reveals comparable CDA
modulations than objects that are integrated on the basis of “amodal”
completion (e.g., due to partial occlusion), given that modal and amodal
completions are associated with comparable behavioral patterns (e.g.,
Chen et al., 2018a) and partially overlapping neural mechanisms (Murray
et al., 2004).
A complementary pattern of results was additionally revealed from the
analysis of the oscillatory amplitudes. For instance, we found a graded,
grouping-strength-related modulation in the lateralized alpha
suppression during the maintenance phase, which was comparable to the
observed PPC variation during initial perceptual processing.
Specifically, the observed differences in the lateralized alpha
amplitudes that we observed were associated mainly with contralateral
variations, that is, they reflect the processing of the task-relevant
stimulus configurations rather than the inhibition of task-irrelevant
placeholders (see also
Bacigalupo & Luck 2019;
Thut et al., 2006; Noonan et al., 2016). Moreover, our findings show
that the lateralized alpha suppression increased as grouping strength
decreased, thus possibly reflecting the ‘effort’ in representing
individual features of less vs. more strongly grouped objects despite a
constant number of to-be-encoded locations. Posterior-occipital alpha
has been suggested to reflect top-down adjustments of attentional
control (e.g., Thut et al., 2006; Murphy et al. 2020; Wang et al., 2019;
2021; Woodman et al., 2022). In agreement with this interpretation, the
variation of lateralized alpha in the current study may index the larger
attentional demands required to process the less (vs. more) grouped
configurations. This may also explain the similarity of the result
pattern revealed for the lateralized alpha suppression to concurrent
variations in the PPC component since both neuronal signatures are
associated with comparable (attentional) processing requirements (albeit
at different levels of processing).
We would thus propose that the CDA and lateralized alpha suppression
might be mapped onto two separable cognitive mechanisms of VWM, relating
to (i) the representation of individual objects and (ii) associated
attentional control processes, respectively (Unsworth et al. 2014; see
also Fukuda et al., 2015). That is, the CDA effect as a function of
grouping might be more consistent with mechanisms necessary for holding
multiple individuated representations in an active state over the
duration of the retention interval. And the increased lateralized alpha
suppression for more ungrouped elements might in turn be directly
associated with the increased attentional control demands for keeping
the individual representations accessible during maintenance when the
displayed stimulus configuration increases in perceptual complexity. In
fact, we found that when quantifying grouping by means of the difference
between grouped and ungrouped configurations, the CDA and the
lateralized alpha amplitude were marginally correlated (r = 0.30,p = 0.075). This indicates that the two neurophysiological
signals are likely manifestations of linked neural processes (see also
van Dijk et al., 2010; Mazaheri & Jensen, 2008; but see Fukuda et al.,
2015), with lower effort and higher precision in representing features
of the grouped configuration as compared to higher effort and lower
precision relating to ungrouped configurations. While being overall
consistent with our current findings, admittedly, this interpretation of
the relation between the CDA and alpha activity is, to a certain extent,
still speculative and it might thus be necessary to consolidate these
interpretations in future studies.
Our current findings may also be related to a theoretical framework that
we proposed previously (Chen et al., 2021a), and according to which
objects might be stored in VWM in terms of a hierarchical structure,
comprising basic feature-level representations and associated
higher-order, object-level representations (see also Brady et al., 2011;
Nie et al., 2017). The representation of information at different levels
in this hierarchy would jointly determine the capacity and quality of
VWM representations. Moreover, depending on the current task, only
to-be-remembered objects and associated features would be encoded and/or
represented (Bocincova & Johnson, 2019; Chen et al., 2021a; Serences et
al., 2009; Woodman & Vogel, 2008). In this view, the encoding of
grouping-relevant (orientation) features would give rise to the encoding
of the superordinate grouped object (which would in turn be represented
at a higher level in the hierarchical memory structure, and which is
reflected by the effects seen in the early PPC). The superordinate
object representation in turn strengthens the representation of the
grouping-relevant features (via reciprocal-feedforward/feedback
connections). Moreover, the superordinate object representation also
enhances the attentional selection (as reflected by the enhanced N1pc
and N2pc) of both grouping-relevant and -irrelevant features at the
basic level of representation via a feedback connection as the complete
object is brought into the “focus of attention” (see, e.g., Oberauer
& Hein, 2012; Souza & Oberauer, 2017; Printzlau et al. 2022). In the
subsequent retention stage, the number and precision of feature
representations is improved by the (superordinate) grouped object
representation which might be reflected by variations in the CDA
amplitude, while concurrent alpha activity would index the attentional
requirements to process these maintained objects (see Machizawa et al.,
2012; Wang et al., 2019; 2021; Woodman et al., 2022).