Top-down influences on visual processing

The Receptive Field

Top-down influences take into account the nature of stimulus dependent properties in any sensory cortical area. There is an emerging view that in the early stages of visual cortical processing, rather than doing a local analysis of simple features, neurons can integrate information over large parts of the visual field, and that neurons in these areas can show selectivity for complex stimulus configurations. The integrative properties of cortical neurons are reflected in their selectivity for stimulus context. Contextual influences refer to the ways by which the perceptual qualities of a local feature are affected by surrounding scene elements and the way in which global scene characteristics affect the responses of neurons to local features. They play important roles in perceptual grouping, perceptual constancies, contour integration, surface segmentation and shape recognition. The most profound effects of top-down control are exerted on contextual influences. This has led to a change in our thinking about the role and prevalence of top-down influences across the visual cortical hierarchy, from initial studies suggesting that they are negligible at early stages of cortical processing to current studies showing substantial changes in neural responses with shifts in attention, with expectation and perceptual task.

Understanding how such cognitive influences affect neuronal function requires understanding the character of the receptive field. The visual receptive field is the part of the retina where a stimulus can cause the neuron to respond with a train of action potentials. The characterization of the receptive field is dependent on the nature of the stimulus used to measure it. A simple stimulus, such as an oriented line segment, will activate a neuron over a small part of the visual field (known as the “minimum response field”, which for superficial layer V1 parafoveal receptive fields is in the order of 0.5° in diameter), but similar stimuli outside this area, which by themselves will not activate the neuron, can greatly affect the neuron’s response when presented jointly with a stimulus in the centre of the receptive field. These modulatory influences can be either facilitatory or inhibitory, and the direction and size of the effect depends on the position of the flanking stimulus relative to the receptive field core¹. As a consequence, neurons’ responses are as dependent on the characteristics of global contours and surfaces as they are on the attributes of local features within the minimum response field, and these contextual influences can extend over relatively large regions of the visual field. Contextual influences in areas V1 and V2 have been implicated in intermediate level vision, including contour integration (the assembly of contour elements into global shapes) and surface segmentation (the separation between object surfaces and their backgrounds)^2-8. The extent of these contextual influences varies according to the level of stimulus complexity and attentional state⁹. One has to keep contextual influences in mind when considering the role of top-down influences in altering a neuron’s response properties.

The function of a neuron is also characterized by its tuning to a range of stimuli, such as different line orientations, directions of movement or colors. One can extend this to any stimulus space and determine the responsiveness of a neuron to stimuli in different points within that space. This has been applied to determine a neuron’s selectivity for the shapes of complex objects or for the configuration of complex stimuli consisting of multiple line segments. Beyond examining the shape of a neuron’s tuning, one can use other measures to characterize a neuron’s stimulus selectivity and to relate that selectivity to perception¹⁰. One is mutual information – the degree to which a neuron’s response predicts stimulus identity, quantified in bits. Another is ideal observer analysis, which allows one to relate a neuron’s discriminability in a stimulus space, its “neurometric” curve, or to the animal’s discrimination performance, its “psychometric” curve. Top-down influences also affect these measures of neuronal function, and as a result, change the nature of the information that neurons convey.

The cortical source and circuitry underlying contextual influences have been vigorously debated^{5, 11-17}. We have proposed that long range intrinsic cortical connections provide a substrate for interactions across the visual field, and have a spatial extent and columnar specificity that is consistent with the contextual influences and with the Gestalt rules of perceptual grouping^{5, 11, 12, 14}. Some have argued that these influences originate from higher order cortical areas, based on their timing relative to stimulus onset. It is not clear that timing is a reliable indicator of the source of a signal given the fast conduction velocities of feedback projections. An alternative explanation is that a signal delay is due to the time required for the network to shift from one stable state to another, with foreground and background interactions requiring time to evolve¹⁸. Delayed influences have been seen with stimuli involving texture segmentation and contour saliency^{5, 19}, but for stimulus configurations without complex backgrounds contextual effects have been observed from the onset of responses²⁰ (see Figure 2).

The nature of top-down influences and their effect on receptive field properties

Top-down influences include different forms, such as attention, expectation and perceptual task. They are seen at all stages in the visual hierarchy, including primary visual cortex, and reaching as far back as the lateral geniculate nucleus^{21, 22}. The effect of these influences is to alter receptive field properties and the information carried by neural ensembles. As a consequence, vision can be thought of as an active process, requiring expectation or hypothesis testing in order to interpret the visual scene. Some contextual influences have been proposed to arise from a predictive coding strategy, where higher levels in the cortical hierarchy make predictions about lower-level activity, and some neurons carry an error signal between the prediction and the stimulus generated activity^{23, 24}. Top-down influences assume a number of forms, and there is a rich amount of information conveyed from higher order to lower order areas:

Dynamic encoding of information at the network level

A useful way to think about the effect of top-down influences is in terms of the information they convey and impart upon their target neurons. Information theory provides a measure of the extent an ideal observer can categorize a stimulus based on the spike count from a recorded neuron during one trial. Top-down influences affect neuronal tuning in a way that enables neurons to carry more information about the stimulus being discriminated. Neurons can increase the degree of modulation of their responses over a set of stimuli, making these responses more informative about stimulus identity.

The idea that a neuron is an adaptive processor, changing the calculation it performs in accordance to the top-down instruction received from higher order cortical areas, has attendant with it that the neuron’s line label is not fixed. The line label idea suggests that when a neuron fires it is signaling the presence of a stimulus possessing the neuron’s preferred attribute (orientation preference, for example), and the strength of its firing indicates the closeness of the stimulus to that attribute. But if the top-down signal causes neurons to change the meaning of the information they carry, then these neurons are effectively changing their line label. How can this not distort the analysis of the visual image if neurons are constantly changing their function? The answer lies in the fact that the higher order areas sent the instruction for these neurons to perform a particular calculation, so the return signal is “interpreted” by these areas as the result of that calculation, and is not confused with other operations those neurons perform.

Beyond the effect of top-down influences on the functional properties of individual neurons, neuronal ensembles can be induced to carry more information by altering their correlation structure, that is, the spatial and temporal distribution of correlated activity over the network of neurons within and across cortical areas. Neurons are variable in their responses to a given stimulus, and as more neurons participate in encoding the stimulus, the variability can be averaged out to provide better signal to noise ratios. But this depends on the ability of neurons to be independent from one another. The optimal information content would require zero or low noise correlations. There is, however, a significant amount of noise correlation^88-91, so a decrease in noise correlations induced by top-down influences would increase the amount of information encoded by the neuronal ensemble^92-95. Decorrelation in the trial to trial variability of responses can allow groups of neurons to average out this variability and improve the signal to noise ratio. This benefit depends on whether neurons are similarly tuned as noise correlation between differently tuned neurons can increase coding efficiency^{92, 93, 96}.

Attention and perceptual learning have been shown to reduce noise correlations, although this has been an area of some debate^97-100. Even more task-specific effects are seen on noise correlations between cortical sites that are relevant to the task being performed, and these changes are larger than those associated with merely attending to the stimulus¹⁰¹. In area MT, noise correlations between a pair of neurons receiving identical visual stimuli can either increase or decrease depending on which of two orthogonal axes the monkey is cued to perform a motion detection task¹⁰².

Top-down influences go well beyond specifying the locus of spatial attention and changing neuronal firing rates. The recurrent pathways that convey these influences must be capable of conveying much more information than a locus to be attended. By the same token, top-down influences cause neurons at the antecedent stages in the cortical hierarchy to alter the nature of the information in their signals. This is not simply a matter of gain control, but involves alterations in tuning that enable neurons to carry more information about stimulus components that are relevant to the task at hand, to take on selectivity for features that are components of expected objects, and to maintain a stable representation of the world in the face of continual eye movements. The increase in task relevant information is contributed in part by the changes in tuning of individual neurons and in part by the change in the structure of correlations across the neuronal ensemble.

Different forms of top-down influences have been documented in different cortical areas, and these effects are relevant to the functional properties of these areas. But all cortical areas, and even the thalamus, can exhibit profound top-down influences. Based on early findings on the lack of attentional effects in V1, along with findings of strong effects in V4 and MT, it has been suggested that attentional influences get progressively stronger along the visual pathway⁴². However, more recent findings, based on more complex stimuli and behavioral paradigms, have called this idea into question, and have suggested that all areas in the hierarchy are equally subject to top-down influences. It is becoming increasingly evident that attention effects are seen early in the visual pathway^{21, 22, 26, 35, 43, 44, 46, 103}. The way in which these influences are manifest depends on the functional role of each cortical area: contour integration in V1, responses to movement direction in MT, modulation by eye position in parietal areas, and so on.

Circuit mechanisms of top-down control

Many studies on top-down influences have focused on the enhancement or change in gain of responses induced by attention, which is equivalent to the stimulus being increased in contrast^{50, 52, 104, 105}. The influence of attention on stimuli within the receptive field has been described in the ‘biased competition’ model⁴¹. In this model objects in the visual field compete for computing resources, and an object can “win” on the strength of its saliency (“bottom up” attention or pop-out) or behavioral relevance (top-down control). Related to the idea of biased competition is a normalization model of attention, which involves a multiplicative scaling of responses to multiple stimuli in the receptive field, and attention affects the strength of the normalization^{106, 107}. These models assume that attention does not affect the stimulus selectivity of neurons. But top-down influences can alter the information carried in neuronal signals, which is distinct from a gain control. For example, changes in a neuron’s tuning to the specific components of the stimulus that are relevant to the task being performed have been observed^{20, 38}, rather than a generalized increase in response to attended stimuli. Attention can change stimulus selectivity in addition to changing gain of responses¹⁰⁸. It is therefore useful to have a model that can account for the specificity of top-down influences for different contextual components and for a neuron’s ability to select a subset among all its inputs in order to exhibit different functional properties. According to this model, although a neuron receives thousands of inputs from intrinsic connections, only a fraction of these are expressed under a particular behavioral context. Interactions between reentrant connections from higher order cortical areas and intrinsic circuits enable the network to gate the connections that are appropriate for the task at hand, with different functional networks operating under different task conditions. As a consequence neurons multiplex their function in a state-dependent manner, and constitute adaptive processors running different operations under the instruction of feedback from higher order cortical areas¹⁰⁹.

The contextual influences that mediate higher order, complex receptive field properties in V1 involve lateral interactions across a topographically organized region, and they have the consequence of perceptual grouping, such as that involved with linking line segments to global contours. The interactions follow precise geometric rules, showing facilitatory influences for neurons with receptive fields lying along collinear or cocircular contours. This mode of interaction is known as the “association field”¹¹⁰. This is a general entity that has been identified in V1, but that is likely to have an analog in all cortical areas. The idea underlying the association field is a linkage between elements that are systematically represented, topographically, over each cortical area. Lateral interactions between these elements allow perceptual linkage or association of pieces of information. The lateral interactions may be mediated by a plexus of long-range horizontal connections within V1. These connections are formed by pyramidal neurons, whose axons extend for long distances parallel to the cortical surface, and link neurons with widely separated receptive fields^{11, 14, 111-114}. Because of their extent and columnar specificity (they connect neurons of similar orientation preference^{11, 14, 115, 116}), they are ideal conveyors of the contextual influences that enable contour integration¹². Although the horizontal connections provide an anatomical framework for a range of contextual interactions, the observation that these interactions are subject to top-down control suggests that feedback signals can alter the effective connectivity of horizontal connections.

We have proposed that reentrant inputs dynamically modify intrinsic cortical connections, allowing the appropriate associations to be made under different behavioral contexts. A possible reason for the existence of horizontal connections is that they allow such changes in connectivity within the network, as opposed to each cell having a large receptive field generated by a fixed set of feedforward connections. This idea has been implemented in models of cortical circuitry, in which changes in the gain of horizontal connections by feedback allows subsets of neuronal inputs to be selectively expressed¹⁸. It also accounts for the time course of contextual interactions, where delayed components of neuronal responses are due to the time required for the network to move from one stable state to another, rather than a function of the conduction time required to get information from a distant, more central source. Finally, it provides a mechanism for contour integration and saliency¹⁸. The interaction between feedback and horizontal connections also suggests a mechanism for perceptual learning. During the encoding of learned information the recurrent input acquires the appropriate mapping to intrinsic connections, and during the recall of the learned information this relationship allows the appropriate inputs to be gated and the target neuron to assume the appropriate functional properties. In V1 the association field mediates contour integration and saliency, and the top-down input allows for sub-components of the association field to be gated, leading to the manifestation of different shape selectivities. In other areas the association field would be defined by the properties and the kind of information that are topographically mapped in that area, and by the relationship between the long range horizontal connections and that map.

Many of the task and expectation dependent effects described above can be explained by an input selection mechanism. By selecting components of the association field, neurons can express contextual influences that are relevant to the task being performed. A contour detection task enhances collinear interactions and suppresses influences from non collinear elements in the background⁵. A shape discrimination task induces neurons to select collinear influences when the cue is a line and cocircular influences when the cue is a circle⁸. By selection of components of the association field over multiple nodes in the horizontal network, neurons in V1 can take on selectivity for complex shapes, including wave like shapes with reversals in curvature. The selective influence of parallel lines in a 3-line bisection task and collinear lines in a vernier discrimination task²⁰ can be mediated by changing the effective connectivity of task relevant inputs. This idea is supported by an experiment involving recording from an array of electrodes, where the interactions between cortical sites are measured by cross-correlation analysis (based on the relative timing of spikes between pairs of neurons) or coherence between local field potentials (LFPs) measured at different sites. Changing the perceptual task with the identical visual stimulus strongly influences correlation strength. Perceptual grouping tasks enhance LFP coherence between parallel sites in 3-line bisection, and between collinear sites in contour detection. Perceptual segregation decreases LFP coherence between collinear sites as seen in a vernier discrimination task¹⁰¹ (Figure 5). This is similar to the expectation-dependent changes seen in noise correlations¹⁰². Although some measures of coherence suggest that attention decreases cortical interactions¹¹⁷, the effect of top-down influences depends on the nature of the task and the way in which different cortical sites are engaged in the task. Further support of this idea comes from fMRI measures of coupling between distant cortical sites representing separated stimuli in a task requiring integration of the two stimuli¹¹⁸.

Changes in effective connectivity mediated by top-down influences relates to the idea that neural synchrony is the neural code for perceptual grouping and segmentation^119-125, although some studies have failed to confirm this idea^{102, 126-129}. It has been proposed that perceptual grouping is achieved by synchronizing the activity of neurons representing the grouped features^{130, 131}, and that neuronal synchrony plays important roles in sensorimotor integration^132-134. Synchrony in itself may be more a reflection of the dynamic connectivity leading to task dependent alterations in neural tuning rather than the information being carried by the relative timing of action potentials per se. The two may in fact be related, with alterations in effective connectivity underlying the task-dependent changes in tuning. Selective attention can also provide a solution to the “superposition problem”, where contour components belonging to one object have to be associated with one another and perceptually separated from components that belong to the object’s background. The role of attention in synchronization is seen in animals performing a color change detection task, in which there is gamma band synchronization between cortical sites encoding the behaviorally relevant stimulus¹³⁵. Also, top-down influences can affect effective connectivity between cortical areas. Just as attention can increase gamma band synchronization within V4, it increases synchronization between the frontal eye fields and V4^{136, 137}. This idea is supported in human subjects by fMRI based correlations of BOLD background connectivity between cortical areas, which is specific to task and cortical area¹³⁸. It is important to emphasize that top-down influences don’t just alter effective connectivity in general, they can selectively and differentially change the effective connectivity between cortical sites that are task relevant¹⁰¹.

Signals that represent top-down influences are observed following the cue directing the task or expectation and before the stimulus presentation^{99, 101, 139, 140}. The pre-stimulus task-dependent activity suggests that subthreshold signals set the cortical “state” for executing the calculation that is appropriate for the behavioral context, and that a given percept results from the set of states assumed by the entire network of cortical areas. This view contrasts with the traditional idea that perception results from the activity of a small number of cells at the top of the visual hierarchy. Instead, the percept arises from the global set of cortical states and task-specific interactions between multiple cortical areas.

The connectivity mediating top-down instructions is likely to include cortico-cortical feedback connections. For area V1, for example, although the strongest feedback arises from area V2, there are a number of cortical areas that provide direct recurrent input to V1, including those in the ventral pathway, such as areas V4 and IT, and areas in the dorsal pathway, including area MT^{14, 141-143}. The feedback projection from area MT to V1 has been implicated in visual awareness¹⁴⁴. The ventral pathway inputs could provide information about object expectation and the dorsal pathway inputs could provide information about attentional locus or saliency maps. In addition, other areas, such as prefrontal cortex, could provide executive control over a perceptual task, and the sites of transmission from prefrontal cortex to posterior areas depend on the nature of the task⁶⁹. Though prefrontal cortex is not directly connected to V1, it could exert its influence by a cascade of connections descending through the parietal lobe. Other non-cortical sources of recurrent input have been suggested, such as the pulvinar^{145, 146}. Multiple sources are likely to be involved in top-down control, but as indicated above, they must be capable of carrying the richness of information involved in not only spatial attention but expectation and perceptual task. (Figure 1)

Summary

The existence of such a varied array of top-down influences and their profound effect on the functional properties of neurons (as well as on their interactions within neuronal ensembles) raises a host of questions for further investigation: What are the sources of the various types of top-down control and what are the pathways by which this control is exerted? What is the nature of the signal that is conveyed along these recurrent pathways? What are the synaptic and network mechanisms by which feedforward, recurrent and intrinsic cortical connections interact to enable adaptive changes in neuronal function? The challenge is to address these questions in the context of the intact, functioning system and to do so in behaving animals.

By selecting different sets of inputs neurons take on different functions. The source of top-down influences can be widespread, either by direct connections from different cortical areas, or by a cascade of inputs originating from many more areas. In effect a large part of the cerebral cortex can exert influences over individual neurons within a particular area, with multiple descending inputs interacting with intrinsic cortical connections (Figure 6). As such, each neuron is a microcosm of the brain as a whole, with synapses carrying information originating from far flung brain regions. This mode of operation has important implications for our understanding of the cortical mechanisms underlying all sensory modalities and behaviors, and its dysfunction may be the cause of behavioral disorders.

Biographies