{"title": "A Neural Network Model of 3-D Lightness Perception", "book": "Advances in Neural Information Processing Systems", "page_first": 844, "page_last": 850, "abstract": null, "full_text": "A  Neural Network Model of 3-D \n\nLightness Perception \n\nFederal Univ.  of Rio de Janeiro \n\nRio de Janeiro, RJ, Brazil \n\nLuiz Pessoa \n\npessoa@cos.ufrj.br \n\nWilliam D. Ross \nBoston University \nBoston, MA  02215 \n\nbill@cns.bu.edu \n\nAbstract \n\nA  neural  network  model  of 3-D  lightness  perception  is  presented \nwhich  builds  upon  the  FACADE  Theory  Boundary  Contour  Sys(cid:173)\ntem/Feature Contour  System  of Grossberg  and  colleagues.  Early \nratio encoding by  retinal ganglion neurons  as  well  as psychophysi(cid:173)\ncal  results on constancy  across  different  backgrounds (background \nconstancy)  are used to provide functional constraints to the theory \nand  suggest  a  contrast  negation  hypothesis  which  states that ratio \nmeasures  between  coplanar  regions  are  given  more  weight  in  the \ndetermination  of lightness  of the  respective  regions.  Simulations \nof the  model  address  data on  lightness  perception,  including  the \ncoplanar ratio hypothesis,  the  Benary cross,  and White's illusion. \n\n1 \n\nINTRODUCTION \n\nOur everyday visual experience includes surface color constancy.  That is, despite  1) \nvariations in scene lighting and 2)  movement or displacement across visual contexts, \nthe  color  of an  object  appears  to  a  large  extent  to  be  the  same.  Color  constancy \nrefers,  then,  to the fact  that surface  color remains largely constant  despite  changes \nin  the  intensity  and  composition  of the  light  reflected  to  the  eyes  from  both  the \nobject  itself and from  surrounding objects.  This paper  discusses  a  neural  network \nmodel  of 3D  lightness  perception  -\ni.e.,  only  the  achromatic  or  black  to  white \ndimension of surface  color  perception  is  addressed.  More  specifically,  the  problem \nof background  constancy  (see  2  above)  is  addressed  and mechanisms to accomplish \nit in  a  system exhibiting  illumination  constancy  (see  1 above)  are  proposed. \n\nA  landmark  result  in  the  study  of lightness  was  an  experiment  reported  by  Wal(cid:173)\nlach  (1948)  who  showed  that for  a  disk-annulus  pattern,  lightness  is  given  by  the \nratio of disk  and annulus luminances (i.e., independent of overall illumination); the \n\n\fA Neural Network Model of 3-D Lightness Perception \n\n845 \n\nso-called  ratio principle.  In  another study,  Whittle and  Challands  (1969)  had sub(cid:173)\njects  perform  brightness  matches  in  a  haploscopic  display  paradigm.  A  striking \nresult  was  that subjects  always  matched  decrements  to  decrements ,  or  increments \nto increments,  but never  increments to decrements.  Whittle and  Challands' (1969) \nresults  provide  psychophysical  support  to  the  notion  that  the  early  visual system \ncodes  luminance  ratios  and  not  absolute  luminance.  These  psychophysical  results \nare  in  line  with  results  from  neurophysiology  indicating  that  cells  at  early  stages \nof the  visual system encode  local  luminance contrast  (Shapley  and  Enroth-Cugell, \n1984).  Note that lateral inhibition mechanisms are sensitive to local  ratios  and can \nbe  used  as  part of the explanation of illumination constancy. \n\nDespite  the  explanatory  power  of the  ratio  principle,  and  the  fact  that  the  early \nstages of the visual system likely code contrast, several experiments have shown that, \nin  general,  ratios  are  insufficient  to  account  for  surface  color  perception.  Studies \nof background  constancy  (Whittle  and Challands,  1969;  Land and  McCann,  1971; \nArend and Spehar,  1993), of the role of 3-D spatial layout and illumination arrange(cid:173)\nment  on lightness  perception  (e.g. ,  Gilchrist,  1977)  as  well  as  many other  effects, \nargue against the sufficiency  of local contrast measures (e.g., Benary cross,  White 's, \n1979 illusion).  The neural network model presented  here  addresses these  data using \nseveral  fields  of neurally plausible  mechanisms of lateral inhibition and  excitation. \n\n2  FROM  LUMINANCE  RATIOS  TO  LIGHTNESS \n\nThe  coplanar  ratio  hypothesis  (Gilchrist,  1977)  states  that the  lightness  of a  given \nregion  is  determined predominantly in  relation to other  coplanar surfaces,  and  not \nby  equally  weighted  relations  to  all  retinally  adjacent  regions.  We  propose  that in \nthe  determination  of lightness,  contrast  measures  between  non-coplanar  adjacent \nsurfaces  are  partially negated in  order to preserve  background  constancy. \n\nConsider  the Benary  Cross  pattern  (input stimulus in  Fig.  2).  If the gray patch on \nthe  cross  is  considered  to  be  at  the  same depth  as  the  cross ,  while  the  other gray \npatch is taken to be at the same depth as the background (which is  below the cross), \nthe  gray  patch  on  the  cross  should  look  lighter  (since  its  lightness  is  determined \nin  relation  to  the  black  cross),  and  the  other  patch  darker  (since  its  lightness  is \ndetermined  in  relation  to  the  white  background) .  White's  (1979)  illusion  can  be \ndiscussed  in  similar terms  (see  the  input stimulus in Fig.  3). \n\nThe mechanisms presented below implement a process of partial contrast negation in \nwhich the initial retinal contrast  code is  modulated by  depth information such that \nthe retinal contrast consistent  with the depth  interpretation is maintained while the \nretinal  contrast not supported  by  depth is  negated or  attenuated. \n\n3  A  FILLING-IN  MODEL  OF  3-D LIGHTNESS \n\nContrast/Filling-in models propose  that  initial measures  of boundary  contrast fol(cid:173)\nlowed  by spreading  of neural  activity within filling-in  compartments produce  a  re(cid:173)\nsponse  profile  isomorphic  with  the  percept  (Gerrits  &  Vendrik,  1970;  Cohen  & \nGrossberg,  1984;  Grossberg  &  Todorovic,  1988;  Pessoa,  Mingolla,  &  Neumann, \n1995).  In this  paper  we  develop  a  neural  network  model of lightness perception  in \nthe tradition of contrast/filling-in theories.  The neural network developed here is an \nextension  of the  Boundary  Contour  System/Feature  Contour  System  (BCS/FCS) \nproposed  by  Cohen  and  Grossberg  (1984)  and  Grossberg  and  Mingolla  (1985)  to \nexplain 3-D lightness  data. \n\n\f846 \n\nL. PESSOA. W. D. ROSS \n\nA fundamental idea of the BCS/FCS theory is that lateral inhibition achieves illumi(cid:173)\nnation constancy but requires  the recovery  of lightness by the filling-in, or diffusion , \nof featural quality (\"lightness\"  in  our case) .  The final  diffused  activities correspond \nto lightness,  which  is  the outcome of interactions between  boundaries  and featural \nquality,  whereby  boundaries  control  the  process  of filling-in  by  forming  gates  of \nvariable resistance to diffusion . \n\nH ow  can  the  visual  system  construct  3-D  lightness  percepts  from  contrast  measures \nobtained by  retinotopic lateral inhibition? A mechanism that is easily instantiated in \na  neural model and provides  a  straightforward modification to the contrast/filling(cid:173)\nin  proposal of Grossberg  and  Todorovic  (1988)  is  the  use  of depth-gated filling-in. \nThis  can  be  accomplished  through  a  pathway  that  modulates  boundary  strength \nfor  boundaries  between  surfaces  or  objects  across  depth.  The  use  of permeable \nor  \"leaky\"  boundaries  was  also  used  by  Grossberg  and  Todorovic  (1988)  for  2-D \nstimuli.  In the current  usage,  permeability is actively increased at depth boundaries \nto partially negate the contrast effect -\nand \nthus  preserve  lightness  constancy  across  backgrounds.  Figure  1 describes  the  four \ncomputational stages of the system. \n\nsince filling-in proceeds  more freely -\n\nI BOUNDARIES \n\n,...---------, ~ \n\n~ ON/OFF \n~- FILTERING \n\nj \n\nI \n\n'\" I DEPTH I \n\nMAP \n\n~ I RLLlNG-IN I \n\nFigure  1:  Model  components. \n\nStage 1:  Contrast  Measurement.  At this stage  both ON  and OFF  neural  fields \nwith  lateral  inhibitory  connectivity  measure the  strength  of contrast  at  image re(cid:173)\ngions  -\nON  field  is  given  by \n\nin  uniform regions  a  contrast  measurement of zero  results.  Formally,  the \n\ndyi;  _ \n+ )  + \ndt - -aYij + ((3  - Yij  Cij  -\n\n+ \n\n(+ \n)  + \nYij  + 'Y  Eij \n\n(1) \n\nwhere a , (3  and 'Yare constants; ct is the total excitatory input to yi;  and Et;  is the \ntotal inhibitory input to yi; . These  terms denote discrete  convolutions of the input \nIij  with  Gaussian weighting functions,  or kernels.  An  analogous equation specifies \nYi;  for  the OFF  field .  Figure 2 shows  the ON-contrast  minus the  OFF-contrast. \nStage 2:  2-D Boundary Detection.  At Stage 2, oriented odd-symmetric bound(cid:173)\nary  detection  cells  are excited  by  the oriented  sampling of the ON  and OFF  Stage 1 \ncells.  Responses  are  maximal when  ON  activation is  strong  on one  side  of a  cell's \nreceptive  field  and  OFF  activation  is  strong  on  the  opposite  side.  In  other  words, \nthe  cells  are tuned  to ON/OFF  contrast  co-occurrence,  or juxtaposition (see  Pessoa \net  aI.,  1995).  The output at this stage is  the sum of the  activations of such  cells  at \neach location for all orientations.  The output responses  are sharpened and localized \nthrough lateral  inhibition across  space;  an  equation similar to  Equation  1 is  used. \nThe final  output of Stage 2 is  given  by  the signals  Zij  (see  Fig.  2,  Boundaries). \n\nStage 3:  Depth Map.  In  the  current  implementation a  simple scheme  was  em(cid:173)\nployed  for  the  determination  of the  depth  configuration.  Initially,  four  types  of \n\n\fA  Neural Network Model of 3-D Lightness Perception \n\nT-junction cells  detect  such  configurations  in  the  image.  For example, \n\nIij  =  Zi-d ,j  x  Zi+d ,j  x  Zi ,j+d, \n\n847 \n\n(2) \n\nwhere d is  a constant, detects T-junctions, where left , right, and top positions of the \nboundary stage are  active;  similar cells  detect  T-junctions of different  orientations. \nThe  activities  of the  T-junction  cells  are  then  used  in  conjunction  with  boundary \nsignals  to  define  complete  boundaries.  Filling-in  within  these  depth  boundaries \nresults  in  a  depth  map (see  Fig.  2, Depth  Map). \n\nStage  4:  Depth-modulated Filling-in.  In  Stage  4,  the  ON  and  OFF  contrast \nmeasures are allowed to diffuse  across space  within respective filling-in  regions .  Dif(cid:173)\nfusion is blocked  by boundary activations from Stage 2 (see  Grossberg & Todorovic, \n1988, for  details).  The diffusion process  is further  modulated by  depth information. \nThe depth map provides this information; different  activities code different  depths. \nIn  a full  blown implementation of the model, depth information would be obtained \nby the depth segmentation of the image supported by both binocular disparity  and \nmonocular depth  cues. \n\nDepth-modulated  filling-in  is  such  that  boundaries  across  depths  are  reduced  in \nstrength.  This allows a small percentage of the contrast on either side ofthe bound(cid:173)\nary  to  leak  across  it  resulting  in  partial  contrast  negation,  or  reduction,  at  these \nboundaries.  ON  and OFF filling-in domains are used which receive the corresponding \nON  and OFF  contrast  activities from  Stage  1 as  inputs  (see  Fig.  2,  Filled-in). \n\n4  SIMULATIONS \n\nThe  present  model  can  account  for  several  important phenomena,  including 2 - D \neffects  of lightness  constancy  and  contrast  (see  Grossberg  and  Todorovic,  1988). \nThe simulations that follow  address  3 -D lightness effects. \n\n4.1  Benary Cross \n\nFigure  2 shows  the  simulation for  the  Benary  Cross .  The  plotted gray  level  values \nfor  filling-in reflect  the activities of the ON  filling-in domain minus the OFF  domain. \nThe  model  correctly  predicts  that the  patch on  the  cross  appears  lighter  than  the \npatch on  the  background.  This result  is  a  direct  consequence  of contrast  negation. \nThe depth relationships are such that the patch on the cross is at  the same depth as \nthe cross  and the patch on the background is  at the same depth  as  the  background \n(see  Fig.  2,  Depth  Map) .  Therefore,  the  ratio of the  background  to  the  patch  on \nthe  cross  (across  a  depth  boundary)  and  the  ratio  of  the  cross  to  the  patch  on \nthe  background  (also  across  a  depth  boundary),  are  given  a  smaller  weight  in  the \nlightness  computation.  Thus,  the  background  will  have  a  stronger  effect  on  the \nappearance of the patch on the background,  which will appear darker.  At  the same \ntime,  the  cross  will  have  a  greater  effect  on  the  appearance  of the  patch  on  the \ncross,  which  will  appear lighter. \n\n4.2  White's lllusion \n\nWhite 's  (1979)  illusion  (Fig.  3)  is  such  that  the  gray  patches  on  the  black  stripes \nappear lighter than  the gray patches on  the  white stripes.  This effect  is  considered \na  puzzling  violation  of  simultaneous  contrast since  the  contour  length  of the  gray \npatches  is  larger  for  the  stripes  they  do  not  lie  on .  Simultaneous  contrast  would \npredict  that  the  gray  patches  on  the  black  stripes  appear  lighter  than  the ones on \nwhite. \n\n\f848 \n\nL. PESSOA, W.  D. ROSS \n\nI \nL~ \n\nI \nI \n\n- --- I \n\nBoundaries \n\nDepth Map \n\nStimulus \n\nON-OFF Contrast \n\nFilled-in \n\nFigure 2:  Benary Cross.  The filled-in values of the gray patch on the cross are higher \nthan  the  ones  for  the  gray  patch  on  the  background.  Gray  levels  code  intensity; \ndarker  grays  code  lower values,  lighter grays  code  higher values. \n\nFigure  3  shows  the  result  of the  model  for  White's  effect .  The  T-junction  infor(cid:173)\nmation  in  the  stimulus  determines  that  the  gray  patches  are  coplanar  with  the \npatches  they  lie  on.  Therefore,  their  appearance  will  be  determined  in  relation  to \nthe contrast  of their respective  backgrounds.  This is  obtained,  again, through  con(cid:173)\ntrast  modulation,  where  the  contrast  of,  say,  the  gray  patch  on  a  black  stripe  is \npreserved,  while the  contrast of the same patch with  the  white  is  partially negated \n(due  to the  depth arrangement). \n\n4.3  Coplanar Hypothesis \n\nGilchrist (1977) showed that the perception of lightness is  not determined by retinal \nadjacency,  and that  depth  configuration  and  spatial  layout  help  specify  lightness. \nMore specifically, it was proposed that the ratio of coplanar surfaces,  not necessarily \nretinally  adjacent,  determines  lightness,  the  so-called  coplanar  ratio  hypothesis. \nGilchrist was able to convincingly demonstrate this by  comparing the perception of \nlightness  in two equivalent displays  (in  terms of luminance  values),  aside from  the \nperceived depth  relationships in the  displays. \n\nFigure 4 shows computer simulations of the coplanar ratio effect.  The same stimulus \nis  given  as  input  in  two  simulations  with  different  depth  specifications.  In  one \n(Depth  Map  1),  the  depth  map specifies  that the  rightmost patch is  at  a  different \ndepth than the two leftmost patches  which are coplanar.  In  the other  (Depth  Map \n2), the two rightmost patches are coplanar and at a different depth than the leftmost \npatch.  In all, the depth organization alters the lightness of the central region, which \nshould  appear  darker in  the configuration of Depth  Map  1 than the one for  Depth \nMap 2.  For Depth Map 1, since the middle patch is coplanar with a white patch, this \npatch is  darkened  by  simultaneous contrast.  For  Depth  Map  2,  the  middle patch \nwill  be  lightened  by  contrast  since  it is  coplanar  with  a  black  patch.  It should  be \nnoted that the depth maps for  the simulations shown in  Fig . 4 were  given  as  input. \n\n\fA Neural Network Model of 3-D Lightness Perception \n\n- ---\n\n- 1 \n\n1 \n\nBoundaries \n\n849 \n\n-, \n\nStimulus \n\nON-OFF Contrast \n\nFilled-in \n\nFigure 3:  White's effect.  The filled-in values of the gray patches on the black stripes \nare  higher  than the ones for  the gray patches on white stripes. \n\nThe  current  implementation cannot  recover  depth  trough  binocular  disparity  and \nonly employs monocular cues  as  in  the previous simulations. \n\n5  CONCLUSIONS \n\nIn  this  paper,  data from  experiments on  lightness  perception  were  used  to extend \nthe BCSjFCS theory of Grossberg  and colleagues to account for  several challenging \nphenomena.  The  model  is  an  initial step  towards  providing  an  account  that  can \ntake into consideration the complex factors involved in 3-D vision -\nsee  Grossberg \n(1994)  for  a  comprehensive  account  of 3-D  vision. \n\nAcknowledgements \n\nThe authors would like to than Alan Gilchrist and Fred Bonato for their suggestions \nconcerning  this  work.  L.  P.  was  supported  in  part by  Air  Force  Office  of Scientific \nResearch  (AFOSR F49620-92-J-0334) and Office  of Naval  Research  (ONR N00014-\n91-J-4100);  W.  R.  was  supported  in part  by  HNC  SC-94-001. \n\nReference \n\nArend ,  L.,  &  Spehar,  B.  (1993)  Lightness,  brightness,  and  brightness  contrast:  2. \n\nReflectance  variation.  Perception  {3  Psychophysics  54 :4576-468. \n\nCohen,  M.,  &  Grossberg,  S.  (1984)  Neural  dynamics  of  brightness  perception: \nFeatures,  boundaries,  diffusion,  and  resonance.  Perception  {3  Psychophysics \n36:428-456. \n\nGerrits,  H.  &  Vendrik,  A.  (1970)  Simultaneous  contrast,  filling-in  process  and  in(cid:173)\n\nformation  processing  in  man's  visual  system.  Experimental  Brain  Research \n11:411-430. \n\n\f850 \n\nL. PESSOA, W. D. ROSS \n\nFilled-in 2 \n\nStimulus \n\nDepth Map  1 \n\nFilled-in 1 \n\nFigure  4:  Gilchrist's coplanarity.  The  Filled-in  values  for  the  middle patch on  top \nare  higher  than on  bottom. \n\nGilchrist,  A.  (1977)  Perceived  lightness  depends  on  perceived  spatial arrangement. \n\nScience  195:185-187. \n\nGrossberg,  S.  (1994)  3-D vision and figure-ground  separation by  visual cortex.  Per(cid:173)\n\nception  &  Psychophysics 55:48-120 . \n\nGrossberg, S., & Mingolla, E. (1985) Neural dynamics of form perception:  Boundary \ncompletion,  illusory  figures,  and  neon  color  spreading.  Psychological  Review \n92:173-211. \n\nGrossberg,  S.,  & Todorovic.  D. (1988).  Neural  dynamics of 1-D  and  2-D brightness \nperception:  A unified  model of classical  and recent  phenomena.  Perception  & \nPsychophysics 43:241-277 . \n\nLand, E., & McCann,  J . (1971).  Lightness and retinex theory. Journal of the  Optical \n\nSociety  of America 61:1-11. \n\nPessoa,  L.,  Mingolla,  E.,  &  Neumann,  H. (1995)  A  contrast- and luminance-driven \nmultiscale network  model of brightness perception.  Vision  Research 35:2201-\n2223. \n\nShapley, R., & Enroth-Cugell, C.  (1984) Visual adaptation and retinal gain controls. \nIn  N.  Osborne  and  G.  Chader  (eds.),  Progress  in  Retinal  Research,  pp.  263-\n346.  Oxford:  Pergamon Press. \n\nWallach,  H. (1948) Brightness constancy  and the nature of achromatic colors.  Jour(cid:173)\n\nnal  of Experimental Psychology  38:  310-324. \n\nWhite,  M.  (1979)  A new  effect  of pattern on  perceived  lightness.  Perception 8:413-\n\n416 . \n\nWhittle,  P.,  & Challands,  P.  (1969)  The  effect  of background  luminance  on  the \n\nbrightness  of flashes .  Vision  Research 9:1095-1110. \n\n\f", "award": [], "sourceid": 1068, "authors": [{"given_name": "Luiz", "family_name": "Pessoa", "institution": null}, {"given_name": "William", "family_name": "Ross", "institution": null}]}