{"title": "Dual Mechanisms for Neural Binding and Segmentation", "book": "Advances in Neural Information Processing Systems", "page_first": 993, "page_last": 1000, "abstract": null, "full_text": "Dual Mechanisms  for  Neural Binding \n\nand  Segmentation \n\nPaul Sajda and  Leif H.  Finkel \nDepartment of Bioengineering and \nInstitute of Neurological  Science \n\nUniversity of Pennsylvania \n\n220  South 33rd  Street \n\nPhiladelphia,  PA. 19104-6392 \n\nAbstract \n\nWe  propose  that  the  binding  and  segmentation  of visual  features \nis  mediated  by  two  complementary  mechanisms;  a  low  resolu(cid:173)\ntion,  spatial-based,  resource-free  process  and  a  high  resolution, \ntemporal-based,  resource-limited  process.  In the visual  cortex,  the \nformer  depends  upon  the orderly  topographic organization in  stri(cid:173)\nate  and  extrastriate  areas  while  the  latter  may  be  related  to  ob(cid:173)\nserved  temporal  relationships  between  neuronal  activities .  Com(cid:173)\nputer  simulations  illustrate  the  role  the  two  mechanisms  play  in \nfigure/ ground discrimination,  depth-from-occlusion,  and the vivid(cid:173)\nness  of perceptual completion. \n\n1  COMPLEMENTARY BINDING  MECHANISMS \n\nThe  \"binding  problem\"  is  a  classic  problem  in  computational  neuroscience  which \nconsiders how  neuronal activities are grouped to create mental representations.  For \nthe  case  of visual  processing,  the  binding  of neuronal  activities  requires  a  mecha(cid:173)\nnism  for  selectively  grouping fragmented  visual  features  in  order  to  construct  the \ncoherent representations (i.e.  objects)  which we  perceive.  In this paper we  argue for \nthe  existence  of two  complementary  mechanisms  for  neural  binding,  and  we  show \nhow  such mechanisms  may operate  in  the  constructiO:l  of intermediate-level visual \nrepresentations. \n\n993 \n\n\f994 \n\nSajda and Finkel \n\nOrdered  cortical  topography  has  been  found  in  both striate  and extrastriate  areas \nand  is  believed to be  a  fundamental organizational principle  of visual  cortex.  One \nfunctional  role  for  this  topographic  mapping  may  be  to  facilitate  a  spatial-based \nbinding  system.  For  example,  active  neurons  or  neural  populations  within  a  cor(cid:173)\ntical  area  could  be  grouped  together  based  on  topographic  proximity  while  those \nin  different  areas  could  be  grouped  if  they  lie  in  rough  topographic  register.  An \nadvantage  of this  scheme  is  that  it  can  be  carried  out  in  parallel  across  the  visual \nfield.  However, a spatial-based mechanism will  tend to bind overlapping or occluded \nobjects  which  should  otherwise  be  segmented.  An  alternative  binding  mechanism \nis  therefore  necessary for  binding and segmenting overlapping objects and surfaces. \n\nTemporal  binding  is  a  second  type  of  neural  binding.  Temporal  binding  differs \nfrom spatial binding in  two essential ways;  1)  it operates at  high  spatial resolutions \nand  2)  it  binds  and  segments in  the temporal  domain,  allowing for  the  coexistence \nof  multiple  objects  in  the  same  topographic  region.  Others  have  proposed  that \ntemporal  events,  such  as  phase-locked  firing  or  I  oscillations  may  play  a  role  in \nneural  binding  (von  der  Malsburg,  1981;  Gray  and  Singer,  1989,  Crick  and  Koch, \n1990).  For purposes  of this discussion,  we  do  not consider the specific  nature of the \ntemporal events underlying neural binding, only that the binding itself is  temporally \ndependent.  The  disadvantage  of  operating  in  the  temporal  domain  is  that  the \nbiophysical  properties  of  cortical  neurons  (e.g.  membrane  time  constants)  forces \nthis  processing  to  be  resource-limited-only  a  small  number  of objects  or  surfaces \ncan  be  bound  and  segmented simultaneously. \n\n2  COMPUTING INTERMEDIATE-LEVEL VISUAL \nREPRESENTATIONS:  DIRECTION  OF  FIGURE \n\nWe  consider  how  these  two  classes  of  binding  can  be  used  to  compute  context(cid:173)\ndependent  (non-local)  characteristics  about  the  visual  scene.  An  example  of  a \ncontext-dependent scene  characteristic  is  contour  ownership  or  direction  of figure. \nDirection  of figure  is  a  useful  intermediate-level  visual  representation  since  it  can \nbe  used  to organize  an  image  into a  perceptual scene  (e.g.  infer  relative  depth and \nlink  segregated  features).  Figure  lA  illustrates  the  relationship  between  contours \nand  surfaces  implied  by  direction of figure. \n\nWe  describe  a  model  which  utilizes  both spatial and  temporal  binding  to  compute \ndirection  of figure  (DOF).  Prior  to  computing  the  DOF,  the  surface  contours  in \nthe  image  are  extracted.  These  contours  are  then  temporally  bound  by  a  process \nwe  call  \"contour  binding\"  (Finkel  and  Sajda,  1992).  In  the  model,  the  temporal \nproperties  of  the  units  are  represented  by  a  temporal  binding  value.  We  will  not \nconsider  the  details  of this  process  except  to say  that  units  with  similar  temporal \nbinding  values  are  bound together  while  those  with  different  values are  segmented. \nIn  vivo,  this  temporal  binding value  may  be  represented  by  phase  of neural firing, \noscillation frequency,  or some other specific  temporal property of neuronal activity. \n\nThe  DOF  is  computed  by  circuitry  which  is  organized  in  a  columnar  structure, \nshown in  figure  2A.  There  are  two  primary  circuits  which  operate  to  compute  the \ndirection  of figure;  one  being  a  temporal-dependent/spatial-independent  (TDSI) \ncircuit  selective  to  \"closure\",  the  other  a  spatial-dependent/temporal-independent \n\n\fDual Mechanisms for Neural Binding and Segmentation \n\n995 \n\nclosure \n\nsimilarity & \nproximity \n\n, \n\nf \n\n....  --\n~\\ /  -\n/ \"  1',,-I  ' \n\ndirection of \nline endings \n\nA \n\nconcavities \n\nB \n\nFigure 1:  A  Direction of figure  as  a surface representation.  At point (1)  the contour \nbelongs  to  the  surface  contour of region  A  and  therefore  A  owns the  contour.  This \nrelationship  is  represented locally  as  a  \"direction of figure\"  vector  pointing toward \nregion  A.  Additional  ownership  relationships  are  shown for  points  (2)  and  (3).  B \nCues  used  in  determining direction of figure . \n\n(SDTI) circuit selective to  \"similarity and proximity\".  There are also two secondary \ncircuits  which playa transient role  in  determining direction of figure.  One is  based \non  the  observation that concave segments  bounded by discontinuities  are  a  cue  for \nocclusion  and  ownership,  while  the  other  considers  the  direction  of line  endings \nas  a  potential  cue.  Figure  IB  summarizes  the  cues  used  to  determine  direction \nof figure.  In  this  paper,  we  focus  on  the  TDSI  and  SDTI  circuits  since  they  best \nillustrate  the nature of the  dual binding mechanisms.  The perceptual consequences \nof attributing  closure  discrimination  to  temporal binding  and similarity/proximity \nto spatial binding is  illustrated  in  figure  3. \n\n2.1  TDSI CIRCUIT \n\nFigure  2B(i)  shows  the  neural  architecture  of the  TDSI  mechanism.  The  activity \nof the  TDSI  circuit  selective  for  a  direction  of figure  a  is  computed  by  comparing \nthe amount of closure on either side of a  contour.  Closure is  computed by summing \nthe  temporal  dependent inputs over  all  directions  i; \n\nT DSl Oi  =  [~Sf(ti) - L Sf- 180o (td] \n\nI \n\nI \n\n1 \n\n0 \n\n(1) \n\nThe brackets ([]) indicate an implicit  thresholding (if x  < 0 then [xl  =  0,  otherwise \n[xl  = x)  and  Si(ti)  and  sf- 180\u00b0 (td  are  the  temporal  dependent  inputs,  computed \nas; \n\nsnt;) =  { \n\n: \n\n{ \n\nif \n\notherwise \n\n(Sj  > ST) \n\nand \n\n((ti  - ~t) < tj  < (ti  + ~t)) \n\n(2) \n\n\f996 \n\nSajda and Finkel \n\nTOSI \n\nson \n\n.... \ntoIfrom \n~  other \n\nDOFcolumns \n\nex - 1800 \n\nt  , \n\nto/from \n\ncontour binding \n\nA \n\n(i) \n\n(ii) \n\nB \n\nFigure  2:  A  Divisions,  inputs,  and  outputs  for  a  DOF  column.  B  The  two  pri(cid:173)\nmary  circuits  operating  to  compute  direction  of figure.  (i)  Top  view  of temporal(cid:173)\ndependent/spatial-independent  (TDSI)  circuit  architecture.  Filled  square  repre(cid:173)\nsents position of a  specific  column in  the network.  Unfilled  squares  represent other \nDOF columns serving as  input to this  column.  Bold  curve corresponds to a surface \ncontour  in  the  input.  Shown  is  the  pattern  of  long-range  horizontal  connections \nconverging  on  the  right  side  of  the  column  (side  ex). \ndependent/temporal-independent (SDTI) circuit architecture.  Shown is the pattern \nof connections  converging on the right side of the  column  (side  ex). \n\n(ii)  Top  view  of  spatial(cid:173)\n\nwhere  ex  and  ex  - 180 0  represent  the  regions  on  either side  of the  contour,  Sj  is  the \nactivation of a  unit  along  the  direction  i  (For  simulations i  varies  between  0 0  and \n315 0  by  increments  of 45 0 ),  6.t  determines  the  range  of temporal  binding  values \nover which the column will integrate input, and ST  is  the activation threshold.  The \ntemporal  dependence  of this  circuit  implies  that  only  those  DOF  columns  having \nthe same  temporal  binding value  affect  the closure  computation. \n\n2.2  SDTI  CIRCUIT \n\nFigure 2B(ii)  illustrates the  neural architecture of the SDTI mechanism.  The SDTI \ncircuit  organizes  elements  in  the  scene  based  on  \"proximity\"  and  \"similarity\"  of \norientation.  Unlike  the  TDSI  circuit  which  depends  upon  temporal  binding,  the \nSDTI circuit uses spatial binding  to access  information  across  the network. \n\nActivity  is  integrated  from  units  with  similar  orientation  tuning  which  lie  in  a \ndirection  orthogonal  to the  contour (i.e.  from parallel line segments).  The  activity \nof the  SDTI  circuit  selective  for  a  direction  of figure  ex  is  computed  by  comparing \ninput from  similar orientations on  either side  of a  contour; \n\nSDTl Oi  = _1_ (2: sf(Od - 2: sr- 180o (Od) \n\nSmax\n\n(3) \n\n. \nt \n\n. \nI \n\nwhere  Smax  is  a  constant for  normalizing  the  SDTI  activity  between  0  and  1  and \nsf (OJ)  and  sf- 1800 (Od  are  spatial  dependent  inputs  selective  for  an  orientation  0, \n\n\fDual Mechanisms for Neural Binding and Segmentation \n\n997 \n\nA \n\nB \n\nFigure  3:  A  The model predicts  that  a  closed  figure  could  not  be  discriminated  in \nparallel search since its detection depends on resource-limited temporal binding.  B \nConversely, proximal parallel segments are predicted to be discriminated in  parallel \nsearch  due  to  resource-free spatial binding. \n\ncomputed  as; \n\n(4) \n\nwhere  ex  and  ex  - 180\u00b0  represent  the  regions  on either side  of the  contour,  ()  is  the \norientation of the  contour,  i  is  the  direction from  which  the  unit receives  input,  Cij \nis  the connection strength  (Cij  falls  off as  a gaussian with distance),  and  Sj(x, y, (}j) \nis  the  activation  of  a  unit  along  the  direction  i  which  is  mapped  to  retinotopic \nlocation  (x, y)  and selective for  an  orientation (}j  (For simulations i  varies  between \nthe following  three angles;  1- (}i,1- ((}i  -45\u00b0), 1- ((}i +45\u00b0)).  Since the efficacy  of the \nconnections,  Cij,  decrease  with  distance,  columns  which  are  further  apart  are  less \nlikely  to  be  bound  together.  Neighboring  parallel  contours  generate  the  greatest \nactivation and the  circuit  tends  to discriminate the region  between the  two parallel \ncontours as  the figure. \n\n2.3  COMPUTED  DOF \n\nThe  activity of a  direction  of figure  unit  representing a  direction  ex  is  given  by  the \nsum of the four  components; \n\nDOF Ci  =  C1(TDSl Ci )  + C2(SDTrl:) + C3 (CON Ci )  + C4 (DLE Ci ) \n\n(5) \nwhere the constants define  the contribution of each cue to the computed DOF. Note \nthat in this paper  we  have  not considered  the mechanisms for  computing the  DOF \ngiven  the  two  secondary  cues  (concavities  (CO N Ci )  and  direction  of line  endings \n(DLE Ci )) .  The  DOF activation is  computed for  all  directions  ex  (For simulations ex \nvaries  between 0\u00b0  and  315\u00b0  by  increments of 45\u00b0)  with  the  direction  producing the \nlargest  activation representing the  direction  of figure. \n\n3  SIMULATION  RESULTS \n\nThe  following  are  simulations  illustrating  the  role  the  dual  binding  mechanisms \nplay in perceptual organization.  All simulations were carried out using  the  NEXUS \nNeural  Simulation  Environment  (Sajda and Finkel,  1992). \n\n\f998 \n\nSajda and Finkel \n\n100  ~ \n>l \n&. \n~ '\" \n\"C u: \n'\" \n\n60 \n\nX \n\n60  0 \n\nC \n\nA \n\nB \n\nFigure  4:  A  128x128  pixel  grayscale  image.  B  Direction  of figure  computed  by \nthe  network.  Direction  of figure  is  shown  as  an  oriented  arrowhead,  where  the \norientation represents the preferred  direction of the DOF unit which is  most active. \nC  Depth of surfaces.  Direction of figure  relationships  (such as  those in  the inset  of \nB)  are used to infer relative depth.  Plot shows % activity of units in the foreground \nnetwork- higher  activity implies  that  the surface is  closer  to  the viewer. \n\n3.1  FIGURE/GROUND  AND  DEPTH-FROM-OCCLUSION \n\nFigure 4A  is  a  grayscale image used  as  input to the network.  Figure 4B  shows the \ndirection  of figure  computed  by the model.  Note  that  though  the surface  contours \nare  incomplete,  the  model  is  still  able  to  characterize  the  direction  of figure  and \ndistinguish  figure/ground  over  most  of the  contour.  This  is  in  contrast  to  mod(cid:173)\nels  proposing diffusion-like mechanisms for  determining figure/ground relationships \nwhich  tend to fail  if complete contour closure is  not realized. \n\nThe model utilizes direction of figure to determine occlusion relationships and strat(cid:173)\nify  objects  in  relative  depth,  results  shown  in  figure  4C.  This  method of inferring \nthe relative depth of surfaces given occlusion is in contrast to traditional approaches \nutilizing  T-junctions.  The obvious  advantage of using  direction  of figure  is  that it \nis  a  context-dependent feature  directly  linked to  the representation of surfaces. \n\n3.2  VIVIDNESS  OF  PERCEPTUAL COMPLETION \n\nOur  previous  work  (Finkel  and  Sajda,  1992)  has  shown  that  direction  of figure  is \nimportant for  completion phenomena, such as  the construction of illusory contours \nand  surfaces.  More  interestingly, our model offers  an explanation for  differences  in \nperceived  vividness  between  different  inducing stimuli.  For example,  subjects  tend \nto  rank  the  vividness  of the illusory  figures  in  figure  5 from  left  to  right,  with  the \nfigure  on  the left  being  the most  vivid  and that on the  right  the  least. \n\nOur model accounts for this effect in terms of the magnitude of the direction of figure \nalong  the illusory  contour.  Figure  6 shows  the  individual components contributing \nto the direction  of figure.  For  a  typical inducer, such as  the pacman in  figure  6, the \nTDSI and SDTI circuits tend to force the direction of figure of the L-shaped segment \nto region  1 while the concavity/convexity transformation tries to force the direction \nof figure  of the  segment  to  be  toward  region  2.  This  transformation  transiently \noverwhelms  the  TDSI  and  SDTI  responses,  so  that  the  direction  of figure  of  the \nL-shaped segment is  toward region  2.  However,  the TDSI and SDTI activation will \naffect  the  magnitude of the  direction  of figure,  as  shown  in  figure  7.  For example, \n\n\fDual Mechanisms for Neural Binding and Segmentation \n\n999 \n\nr .. \n.... \n\nB \n\nr-., \nL..I \n\nc \n\nA \n\nFigure 5:  Illusory  contour vividness  as  a function  of inducer shape.  Three  types  of \ninducers  are arranged  to generate an illusory square .  A  pacman inducer , B  thick L \ninducer  and  C  thin  L inducer.  Subjects  rank  the  vividness  of the  illusory  squares \nfrom left  to right  ((A)  > (B)  > (C)). \n\nC?C?Q \n\nconcavity/convexity \n\ncomponent \n\nTDSI \n\ncomponent \n\nSDn \n\ncomponent \n\nFigure  6:  Processes  contributing to  the  direction  of figure  of the  L-shaped  contour \nsegment.  The  TDSI  and  SDTI  circuits  assign  the  contour  to  region  1,  while  the \nchange of the concavity to a  convexity assigns  the segment  to region  2. \n\n,. \nii \nf!. \n\n.' \n\nItll'\n\ni \n\n-:.:  .14 \n\n:, ':::::::::;:::::::1 \n:~ \n.\" ... \nii \n\" \nE \nii \nIi \n::  +E \n\u2022\u2022 ...... \n\n.5 \n\nA \n\n\".  r \n\nB \n\nr \n\nFigure  7:  A  Activity  of SDTI  units  for  the  upper  left  inducer  of each  stimulus, \nwhere  the  area of each square  is  proportional to  unit  activity.  The  SDTI  units  try \nto  assign  the  L-shaped  segment  to  the  region  of the  pacman.  Numerical  values \nindicates  the  magnitude  of the  SDTI  effect.  B  Magnitude  of  direction  of  figure \nalong  the  L-shaped segment  as  a function  of inducer  shape.  The direction  of figure \nin  all  cases  is  toward the region  of the illusory square. \n\n\f1000 \n\nSajda and Finkel \n\nthe weaker the activation of the TDSI and SDTI circuits, the stronger the activation \nof the  DOF units assigning  the L-shaped segment to region  2. \n\nReferring  back  to the inducer  types  in figure  5,  one  can see  that  though  the  TDSI \ncomponent is the same for  all three inducers (i.e.  all three generate the same amount \nof closure)  the  SDTI  contribution  differs,  shown  quantitatively  in  figure  7 A.  The \ncontribution of the SDTI circuit is  greatest for  the  thin L inducers and least for  the \npacmen inducers-the L-shaped segments for  the pacman stimulus are more strongly \nowned by  the surface of the illusory square than those for  the thin L inducer.  This \nis  illustrated in figure  7B,  a  plot of the magnitude of the direction of figure  for  each \ninducer  configuration.  This  result  can  be  interpreted  as  the  model's  ordering  of \nperceived vividness,  which is  consistent  with  that of human observers. \n\n4  CONCLUSION \n\nIn  this paper we  have  argued for  the  utility of binding  neural activities in  both the \nspatial  and  temporal  domains.  We  have  shown  that  a  scheme  consisting  of these \ncomplementary mechanisms can be  used  to compute context-dependent scene char(cid:173)\nacteristics,  such  as  direction  of figure.  Finally,  we  have illustrated  with  computer \nsimulations  the  role  these  dual binding  mechanisms  play  in  accounting for  aspects \nof figure/ ground perception, depth-from-occlusion,  and perceptual vividness of illu(cid:173)\nsory contours and surfaces.  It is  interesting to speculate on the relationship between \nthese  complementary  binding mechanisms  and  the  traditional  distinction  between \npreattentive and  attentional perception. \n\nAcknowledgements \n\nThis work is supported by grants from ONR (N00014-90-J-1864, N00014-93-1-0681), \nThe Whitaker  Foundation,  and The  McDonnell-Pew  Program in  Cognitive  Neuro(cid:173)\nSCIence. \n\nReferences \n\nF. Crick and C.  Koch.  Towards a neurobiological theory of consciousness.  Seminars \nin  Neuroscience,  2:263-275,  1990. \n\nL.H.  Finkel  and  P.  Sajda.  Object  discrimination  based  on  depth-from-occlusion. \nNeural  Computation,  4(6):901-921,1992. \n\nC.  M.  Gray  and  W.  Singer.  Neuronal  oscillations  in  orientation  columns  of  cat \nvisual  cortex.  Proceedings  of the  National Academy  of Science  USA,  86:1698-1702, \n1989. \n\nP.  Sajda and  L.  Finkel.  NEXUS:  A  simulation  environment for  large-scale  neural \nsystems.  Simulation,  59(6):358-364,  1992. \n\nC.  von der Malsburg.  The correlation theory of brain function.  Technical Report In(cid:173)\nternal  Rep.  No.  81-2,  Max-Plank-Institute for  Biophysical Chemistry,  Department \nof Neurobiology,  Gottingen,  Germany,  1981. \n\n\f", "award": [], "sourceid": 752, "authors": [{"given_name": "Paul", "family_name": "Sajda", "institution": null}, {"given_name": "Leif", "family_name": "Finkel", "institution": null}]}