{"title": "Anatomical origin and computational role of diversity in the response properties of cortical neurons", "book": "Advances in Neural Information Processing Systems", "page_first": 117, "page_last": 124, "abstract": null, "full_text": "Anatomical origin and computational role \nof diversity in the response properties of \n\ncortical neurons \n\nKalanit Grill Spectort \nRafael Malacht \nDepts of t Applied Mathematics and Computer Science and tN eurobiology \n\nShimon Edelmant \n\nThe Weizmann Institute of Science \n\nRehovot 76100, Israel \n\n{kalanit.edelman. malach }~wisdom . weizmann .ac.il \n\nAbstract \n\nThe maximization of diversity of neuronal response properties has been \nrecently suggested as an organizing principle for the formation of such \nprominent features of the functional architecture of the brain as the corti(cid:173)\ncal columns and the associated patchy projection patterns (Malach, 1994). \nWe show that (1) maximal diversity is attained when the ratio of dendritic \nand axonal arbor sizes is equal to one, as found in many cortical areas \nand across species (Lund et al., 1993; Malach, 1994), and (2) that maxi(cid:173)\nmization of diversity leads to better performance in systems of receptive \nfields implementing steerable/shiftable filters, and in matching spatially \ndistributed signals, a problem that arises in many high-level visual tasks. \n\n1 Anatomical substrate for sampling diversity \n\nA fundamental feature of cortical architecture is its columnar organization, mani(cid:173)\nfested in the tendency of neurons with similar properties to be organized in columns \nthat run perpendicular to the cortical surface. This organization of the cortex was ini(cid:173)\ntially discovered by physiological experiments (Mouncastle, 1957; Hubel and Wiesel, \n1962), and subsequently confirmed with the demonstration of histologically defined \ncolumns. Tracing experiments have shown that axonal projections throughout the \ncerebral cortex tend to be organized in vertically aligned clusters or patches. In par(cid:173)\nticular, intrinsic horizontal connections linking neighboring cortical sites, which may \nextend up to 2 - 3 mm, have a striking tendency to arborize selectively in preferred \nsites, forming distinct axonal patches 200 - 300 J.lm in diameter. \n\nRecently, it has been observed that the size of these patches matches closely \nthe average diameter of individual dendritic arbors of upper-layer pyramidal cells \n\n\f118 \n\nKalanit Grill Spector, Shimon Edelman, Rafael Malach \n\n.... \n\n2&00 \n\n10(0 \n\n: : \n\n/ '\" ! \n\n&00 , \n\n~._._._;~ .. _._. r::~ ._. \n\n\\ \n\n~ ~ ~ ~ ~ ~ N \n\n,*,*\".\",p.d hm pa~ \n\n) \n100 \n\nBO \n\n80 \n\n'0 \n\n0:2 \n\n0\" \n\n0 S \n\n0.8 \n\nt \n\n1 :2 \n\n1 .. \n\n1 S \n\nt IS \n\nr \u2022\u2022 o betiIJMn I'IWI'ot'I .. d ptIId! \n\nFigure 1: Left: histograms of the percentage of patch-originated input to the neurons, \nplotted for three values of the ratio r between the dendritic arbor and the patch \ndiameter (0,5, 1.0, 2.0). The flattest histogram is obtained for r = 1.0 Right: the \ndi versity of neuronal properties (as defined in section 1) vs. r. The maximum is \nattained for r = 1.0, a value compatible with the anatomical data. \n\n(see Malach, 1994, for a review). Determining the functional significance of this \ncorrelation, which is a fundamental property that holds throughout various cortical \nareas and across species (Lund et al., 1993), may shed light on the general principles \nof operation of the cortical architecture. One such driving principle may be the \nmaximization of diversity of response properties in the neuronal population (Malach, \n1994). According to this hypothesis, matching the sizes of the axonal patches and the \ndendritic arbors causes neighboring neurons to develop slightly different functional \nselectivity profiles, resulting in an even spread of response preferences across the \ncortical population, and in an improvement of the brain's ability to process the variety \nof stimuli likely to be encountered in the environment. 1 \n\nTo test the effect of the ratio between axonal patch and dendritic arbor size on \nthe diversity of the neuronal population, we conducted computer simulations based \non anatomical data concerning patchy projections (Rockland and Lund, 1982; Lund \net al., 1993; Malach, 1992; Malach et al., 1993). The patches were modeled by disks, \nplaced at regular intervals of twice the patch diameter, as revealed by anatomical \nlabeling. Dendritic arbors were also modeled by disks, whose radii were manipulated \nin different simulations. The arbors were placed randomly over the axonal patches, \nat a density of 10,000 neurons per patch. We then calculated the amount of patch(cid:173)\nrelated information sampled by each neuron, defined to be proportional to the area of \noverlap of the dendritic tree and the patch. The results of the calculations for three \n\n1 Necessary conditions for obtaining dendritic sampling diversity are that dendritic arbors cross \nfreely through column borders, and that dendrites which cross column borders sample with equal \nprobability from patch and inter-patch compartments. These assumptions were shown to be valid \nin (Malach, 1992; Malach, 1994). \n\n\fDiversity in the Response Properties of Cortical Neurons \n\n119 \n\nvalues of the ratio of patch and arbor diameters appear in Figure 1. \n\nThe presence of two peaks in the histogram obtained with the arbor/patch ratio \nr = 0.5 indicates that two dominant groups are formed in the population, the first \nreceiving most of its input from the patch, and the second - from the inter-patch \nsources. A value of r = 2.0, for which the dendritic arbors are larger than the axonal \npatch size, yields near uniformity of sampling properties, with most of the neurons \nreceiving mostly patch-originated input , as apparent from the single large peak in \nthe histogram. To quantify the notion of diversity, we defined it as diversity \"'< \nI ~; I > -1, where n(p) is the number of neurons that receive p percent of their inputs \nfrom the patch, and < . > denotes average over p. Figure 1, right, shows that \ndiversity is maximized when the size of the dendritic arbors matches that of the \naxonal patches, in accordance with the anatomical data. This result confirms the \ndiversity maximization hypothesis stated in (Malach, 1994). \n\n2 Orientation tuning: a functional manifestation \n\nof sam pIing diversity \n\nThe orientation columns in VI are perhaps the best-known example of functional \narchitecture found in the cortex (Bubel and Wiesel, 1962). Cortical maps obtained \nby optical imaging (Grinvald et al., 1986) reveal that orientation columns are patchy \nrather then slab-like: domains corresponding to a single orientation appear as a \nmosaic of round patches, which tend to form pinwheel-like structures. Incremental \nchanges in the orientation of the stimulus lead to smooth shifts in the position of \nthese domains. We hypothesized that this smooth variation in orientation selectivity \nfound in VI originates in patchy projections, combined with diversity in the response \nproperties of neurons sampling from these projections. The simulations described in \nthe rest of this section substantiate this hypothesis. \n\nComputer simulations. The goal of the simulations was to demonstrate that a \nlimited number of discretely tuned elements can give rise to a continuum of responses. \nWe did not try to explain how the original set of discrete orientations can be formed \nby projections from the LGN to the striate cortex; several models for this step can \nbe found in the literature (Bubel and Wiesel, 1962; Vidyasagar, 1985).2 In setting \nthe size of the original discrete orientation columns we followed the notion of a point \nimage (MacIlwain, 1986), defined as the minimal cortical separation of cells with \nnon-overlapping RFs. Each column was tuned to a specific angle, and located at \nan approximately constant distance from another column with the same orientation \ntuning (we allowed some scatter in the location of the RFs). The RFs of adjacent units \nwith the same orientation preference were overlapping, and the amount of overlap \n\n2ln particular, it has been argued (Vidyasagar, 1985) that the receptive fields at the output of the \nLGN are already broadly tuned for a small number of discrete orientations (possibly just horizontal \nand vertical), and that at the cortical level the entire spectrum of orientations is generated from the \ndiscrete set present in the geniculate projection. \n\n\f120 \n\nKalanit Grill Spector, Shimon Edelman, Rafael Malach \n\n007 \n\nOOS \n\nOOs \n\n00' \n\n! \n\n003 \n\n00' \n\n001 \n\n0 \n0 \n\n.0 \n\n20 \n\n30 \n\nrunber ~ shlttlng fIlters \n\n'\" \n\n3< \n\n32 \n\n30 \n\nII \n0 28 \n! \n126 \ng .. .. 24 \u2022 \n\n22 \n\n20 \n\n'\u00b70 \n\n.0 \n\n20 \n\nSo \n\nSO \n\n70 \n\n.. \n\nso \n\n30 \n\n'\" \n\nnunber cI shttng tlMlI'1 \n\nFigure 2: The effects of (independent) noise in the basis RFs and in the steer(cid:173)\ning/shifting coefficients. Left: the approximation error vs. the number of basis RFs \nused in the linear combination. Right: the signal to noise ratio vs. the number of ba(cid:173)\nsis RFs. The SNR values were calculated as 10 loglO (signal energy/noise energy). \nAdding RFs to the basis increases the accuracy of the resultant interpolated RF. \n\nwas determined by the number of RFs incorporated into the network. The preferred \norientations were equally spaced at angles between 0 and 1r. The RFs used in the \nsimulations were modeled by a product of a 2D Gaussian G 1 , centered at rj, with \norientation selectivity G2, and optimal angle Oi: G(r, rj, 0, Oi) = G1(r, rj)G2(O, Oi). \nAccording to the recent results on shiftable/steerable filters (Simon celli et al., \n1992), a RF located at ro and tuned to the orientation ,po can be obtained by a linear \ncombination of basis RFs, as follows: \n\nG(r, ro, 0, ,po) \n\nM-IN-I \n\nL L bj(ro)ki(,po)G(r,rj,O,Oi) \nj=O \nM-I \n2: bj(ro)G1(r, rj) 2: ki(,po)G2(O,Oi) \n\ni=O \n\nN-I \n\nj=O \n\ni=O \n\n(1 ) \n\nFrom equation 1 it is clear that the linear combination is equivalent to an outer \nproduct of the shifted and the steered RFs, with {ki(,pO)}~~1 and {bj(ro)}~~l de(cid:173)\nnoting the steering and shifting coefficients, respectively. Because orientation and \nlocalization are independent parameters, the steering coefficients can be calculated \nseparately from the shifting coefficients. The number of steering coefficients depends \non the polar Fourier bandwidth of the basis RF, while the number of steering filters \nis inversely proportional to the basis RF size (Grill-Spector et al., 1995). In the pres(cid:173)\nence of noise this minimal basis has to be extended (see Figure 2). The results ofthe \nsimulation for several RF sizes are shown in Figure 3, left. As expected, the number \nof basis RFs required to approximate a desired RF is inversely proportional to the \n\n\fDiversity in the Response Properties of Cortical Neurons \n\n121 \n\nThe dependency oI1he nlnber d RF9 a1 the venene. \n\n'. \n\n,5 \n\n2 \n\nvan.,08 \n\n25 \n\n35 \n\nFigure 3: Left: error of the steering/shifting approximation for several basis RF sizes. \nRight: the number of basis RFs required to achieve a given error for different sizes of \nthe basis RFs. The dashed line is the hyperbola num RFs x size = const. \n\nsize of the basis RFs (Figure 3, right). \n\nSteerability and biological considerations. The anatomical finding that the \ncolumnar \"borders\" are freely crossed by dendritic and axonal arbors (Malach, 1992), \nand the mathematical properties of shiftable/steerable filters outlined above suggest \nthat the columnar architecture in VI provides a basis for creating a continuum of RF \nproperties, rather that being a form of organizing RFs in discrete bins. Computation(cid:173)\nally, this may be possible if the input to neurons is a linear combination of outputs \nof several RFs, as in equation 1. The anatomical basis for this computation may \nbe provided by intrinsic cortical connections. It is known that long-range (I\"V 1 mm) \nconnections tend to link cells with like orientation preference, while the short-range \n(I\"V 400 J.lm) connections are made to cells of diverse orientation preferences (Malach \net al., 1993). We suggest that the former provide the inputs necessary to shift the po(cid:173)\nsition of the desired RF, while the latter participate in steering the RF to an arbitrary \nangle (see Grill-Spector et al., 1995, for details). \n\n3 Matching with patchy connections \n\nMany visual tasks require matching between images taken at different points in space \n(as in binocular stereopsis) or time (as in motion processing). The first and foremost \nproblem faced by a biological system in solving these tasks is that the images to be \ncompared are not represented as such anywhere in the system: instead of images, \nthere are patterns of activities of neurons, with RFs that are overlapping, are not \nlocated on a precise grid, and are subject to mixing by patchy projections in each \nsuccessive stage of processing. In this section, we show that a system composed of \nscattered RFs with smooth and overlapping tuning functions can, as a matter of \n\n\f122 \n\nKalanit Grill Spector, Shimon Edelman, Rafael Malach \n\nfact, perform matching precisely by allowing patchy connections between domains. \nMoreover, the weights that must be given to the various inputs that feed a RF carrying \nout the match are identical to the coefficients that would be generated by a learning \nalgorithm required to capture a certain well-defined input-output relationship from \npairs of examples. \n\nDOMAIN A \n\nFigure 4: Unit C receives patchy input from areas A and B which contain receptors \nwith overlapping RFs. \n\nConsider a unit C, sampling two domains A and B through a Gaussian-profile \ndendritic patch equal in size to that of the axonal arbor of cells feeding A and B \n(Figure 4). The task faced by unit C is to determine the degree to which the activity \npatterns in domains A and B match. Let