{"title": "Parallel analog VLSI architectures for computation of heading direction and time-to-contact", "book": "Advances in Neural Information Processing Systems", "page_first": 720, "page_last": 726, "abstract": null, "full_text": "Parallel analog  VLSI  architectures for \ncomputation of heading direction and \n\ntime-to-contact \n\nGiacomo Indiveri \n\ngiacomo@klab.caltech.edu \n\nJorg Kramer \n\nkramer@klab .caltech.edu \n\nChristof Koch \n\nkoch@klab.caltech.edu \n\nDivision of Biology \n\nCalifornia Institute of Technology \n\nPasadena, CA  91125 \n\nAbstract \n\nWe  describe  two  parallel analog VLSI  architectures  that integrate \noptical flow  data obtained from  arrays of elementary velocity  sen(cid:173)\nsors to estimate heading direction and time-to-contact.  For heading \ndirection  computation,  we  performed  simulations to  evaluate  the \nmost important qualitative properties of the optical flow  field  and \ndetermine the best  functional operators for  the  implementation of \nthe  architecture.  For  time-to-contact we  exploited  the  divergence \ntheorem  to integrate  data from  all  velocity  sensors  present  in the \narchitecture  and average out  possible errors. \n\n1 \n\nIntroduction \n\nWe  have  designed  analog  VLSI  velocity  sensors  invariant  to  absolute  illuminance \nand  stimulus  contrast  over  large  ranges  that  are  able  to  achieve  satisfactory  per(cid:173)\nformance  in a  wide variety of cases;  yet such  sensors,  due to the intrinsic nature of \nanalog processing,  lack a  high degree  of precision in their output values.  To exploit \ntheir properties  at a  system level,  we  developed  parallel image processing  architec(cid:173)\ntures  for  applications that  rely  mostly on the  qualitative properties  of the  optical \nflow,  rather  than on  the  precise  values  of the  velocity  vectors.  Specifically,  we  de(cid:173)\nsigned  two parallel architectures  that  employ arrays of elementary  motion sensors \nfor  the computation of heading  direction  and time-to-contact.  The application do(cid:173)\nmain that we  took into consideration for  the implementation of such  architectures, \nis the promising one of vehicle navigation.  Having defined the types of images to be \nanalyzed  and the types of processing  to perform, we  were able to use  a priori infor-\n\n\fVLSI Architectures for Computation of Heading Direction and Time-to-contact \n\n721 \n\nmation to  integrate selectively  the sparse  data obtained  from  the  velocity  sensors \nand determine the qualitative properties of the optical flow  field  of interest. \n\n2  The elementary velocity sensors \n\nA  velocity  sensing  element,  that  can  be integrated  into relatively  dense  arrays  to \nestimate  in  parallel  optical  flow  fields,  has  been  succesfully  built  [Kramer  et  al., \n1995].  Unlike  most  previous  implementations of analog  VLSI  motion  sensors,  it \nunambiguously  encodes  1-D  velocity  over  considerable  velocity,  contrast,  and  il(cid:173)\nluminance  ranges ,  while  being  reasonably  compact.  It implements  an  algorithm \nthat  measure'3  the  time of travel  of features  (here  a  rapid  temporal  change  in  in(cid:173)\ntensity)  stimulus between  two  fixed  locations  on  the  chip.  In  a  first  stage,  rapid \ndark-to-bright  irradiance  changes  or  temporal  ON  edges  are  converted  into short \ncurrent  pulses.  Each  current  pulse  then  gives  rise  to  a  sharp  voltage  spike  and  a \nlogarithmically-decaying voltage signal  at each  edge  detector  location.  The sharp \nspike from one location is  used  to sample the analog voltage of the slowly-decaying \nsignal from  an adjacent location.  The sampled output voltage encodes  the relative \ntime  delay  of the  two  signals,  and  therefore  velocity,  for  the  direction  of motion \nwhere  the  onset  of the  slowly-decaying  pulse  precedes  the  sampling spike.  In  the \nother direction,  a  lower  voltage is sampled.  Each direction thus requires  a separate \noutput stage. \n\n'05 \n\n~  OIlS \nj \n\" > \n\no. \n\nOil!> \n\nO' \n\nVIIIoaIy (mmsec) \n\n40 \n\n.. \n\nFigure  1:  Output  voltage  of a  motion sensing  element  for  the  preferred  direction \nof motion  of a  sharp  high-contrast  ON  edge  versus  image  velocity  under  incan(cid:173)\ndescent  room illumination.  Each  data point represents  the  average of 5 successive \nmeasurements. \n\nAs implemented with a 2 J.tm CMOS process, the size of an elementary bi-directional \nmotion element  (including 30  transistors  and  8 capacitances)  is  0.045  mm2.  Fig.  1 \nshows  that  the  experimental  data  confirms  the  predicted  logarithmic encoding  of \nvelocity  by  the  analog output  voltage.  The  data was  taken  by  imaging a  moving \nhigh-contrast  ON  edge  onto  the  chip  under  incandescent  room  illumination.  The \ncalibration of the image velocity  in the focal  plane is  set  by the 300  J.tm  spacing of \nadjacent  photoreceptors  on the chip. \n\n3  Heading direction computation \n\nTo simplify the computational complexity of the problem of heading direction detec(cid:173)\ntion we restricted our analysis to pure translational motion , taking advantage of the \n\n\f722 \n\nG.  INDIVERI, 1.  KRAMER, C.  KOCH \n\nfact  that for  vehicle  navigation it is possible to eliminate the rotational component \nof motion using lateral accelerometer measurements from the vehicle.  Furthermore, \nto analyze the computational properties  of the optical flow  for  typical vehicle navi(cid:173)\ngation scenes,  we  performed  software simulations on sequences  of images obtained \nfrom a camera with a 64 x 64  pixel silicon retina placed on a moving truck (courtesy \nof B.  Mathur at Rockwell Corporation).  The optical flow fields have been computed \n\nFigure 2:  The sum of the horizontal components of the optical flow  field  is plotted \non the bottom of the figure.  The presence  of more than one zero-crossing  is  due  to \ndifferent  types  of noise  in  the optical flow  computation (e.g.  quantization errors  in \nsoftware simulations or device mismatch in analog VLSI circuits).  The coordinate of \nthe heading direction is computed as the abscissa of the zero-crossing with maximum \nsteepness  and closest  to the abscissa of the  previously selected  unit. \n\nby  implementing an  algorithm  based  on  the  image  brightness  consia'f)cy  equation \n[Verri  et  al.,  1992]  [Barron  et  al.,  1994].  For the application domain considered  and \nthe  types  of optical  flow  fields  obtained  from  the  simulations,  it  is  reasonable  to \nassume that the direction of heading changes smoothly in time.  Furthermore, being \ninterested  in  determining,  and  possibly  controlling,  the  heading  direction  mainly \nalong  the horizontal axis,  we  can greatly  reduce  the complexity of the  problem by \nconsidering  one-dimensional arrays of velocity sensors.  In such  a  case,  if we  assign \npositive  values  to  vectors  pointing in one  direction  and  negative  values  to  vectors \npointing in  the opposite direction,  the heading direction location will correspond  to \nthe  point  closest  to the  zero-crossing.  Under  these  assumptions,  the  computation \nof the horizontal coordInate of the heading direction has been  carried out using the \nfollowing  functional  operators:  thresholding  on  the  horizontal  components  of the \noptical flow  vectors;  spatial smoothing on the resulting values;  detection and evalu(cid:173)\nation of the steepness of the zero-crossings  present  in the array and finally selection \nof the  zero-crossing  with  maximum steepness.  The  zero-crossing  with  maximum \nsteepness  is  selected  only  if its  position  is  in  the  neighborhood  of the  previously \nselected  zero-crossing.  This  helps  to eliminate errors  due to  noise  and  device  mis(cid:173)\nmatch and assures that the computed heading direction location will shift smoothly \nin  time.  Fig.  2 shows  a  result  of the software  simulations, on  an  image of a  road \n\n\fVLSI Architectures for Computation of Heading Direction and Time-to-contact \n\n723 \n\nwith  a shadow on the left  side. \n\nAll  of  the  operators  used  in  the  algorithm  have  been  implemented  with  analog \ncircuits  (see  Fig.  3  for  a  block  diagram of the  architecture).  Specifically,  we  have \n\nWIIb .. \n\nFigure  3:  Block  diagram of the  architecture  for  detecting  heading  direction:  the \nfirst  layer  of the  architecture  computes  the  velocity  of the  stimulus;  the  second \nlayer  converts  the  voltage  output  of the  velocity  sensors  into  a  positive/negative \ncurrent  performing a threshold operation; the third layer performs a linear smooth(cid:173)\ning  operation  on  the  positive  and  negative  halfs  of the  input  current;  the  fourth \nlayer detects zero-crossings  by comparing the intensity of positive currents from one \npixel with negative currents from the neighboring pixel; the top layer implements a \nwinner-take-all network  with distributed excitation, which selects the zero-crossing \nwith  maximum steepness. \n\ndesigned  test chips in which  the thresholding function  has been implemented using \na  transconductance  amplifier  whose  current  represents  the  output  signal  [Mead, \n1989],  spatial smoothing has  been  obtained using  a  circuit  that separates  positive \ncurrents  and  negative  currents  into  two  distinct  paths  and  feeds  them  into  two \nlayers  of current-mode  diffuser  networks  [Boahen  and  Andreou,  1992],  the  zero(cid:173)\ncrossing  detection  and  evaluation  of its  steepness  has  been  implemented  using  a \nnewly designed circuit block based on a modification of the simple current-correlator \n[Delbriick,  1991],  and  the  selection  of  the  zero-crossing  with  maximum steepness \nclosest  to the previously selected  unit  has  been  implemented using  a  winner-take(cid:173)\nall  circuit  with  distributed  excitation  [Morris  et  al.,  1995].  The schematics of the \nformer three circuits, which implement the top three layers of the diagram of Fig. 3, \nare shown  in Fig. 4. \nFig. 5 shows the output of a test  chip in which  all blocks  up  to the diffuser network \n(without the zero-crossing detection stages)  were  implemented.  The velocity sensor \nlayout was modified to maximize the number of units in the 1-D array.  Each veloc(cid:173)\nity sensor  measures 60pm x 802pm.  On a  (2.2mm)2 size  chip we  were  able to fit  23 \nunits.  The shown results  have been obtained by  imaging on  the chip expanding or \ncontracting  stimuli using  black  and  white  edges  wrapped  around  a  rotating drum \nand  reflected  by  an  adjacent  tilted  mirror.  The  point  of contact  between  drum \nand mirror corresponding  to the simulated heading  direction  has  been  imaged ap(cid:173)\nproximately onto  the  15th  unit  of the  array.  As  shown,  the  test  chip  considered \ndoes not achieve 100% correct performance due to errors that arise mainly from the \npresence  of parasitic capacitors in the modified part of the velocity sensor  circuits; \nnonetheless,  at least from  a qualitative point of view,  the data confirms the results \nobtained from  software simulations and  demonstrates  the validity of the approach \nconsidered. \n\n\f-\n\n-\n\n-\n\n-\n-\n~ \n~ '\" \n\n-\n\n-\n\n-\n\n-\n\n-\n\n-\n\n-\n\n-\n\n-\n\n,.. \n\n- ~- -\n\n-\n\n-\nII  rln \nII  ~ \nII \nII \n\n\"\"'>---t~;-------'~:-----<-'  II  I.. \n\"'.>----t ....... -----i'-----<,..t  II \nII \nII \nII \nII \nII \n\nill \n\n-\n\n-\nII  ~ \nII  cp \nII \nII \nII \n!P II \n\n.. \n\n.. _\"-\n\nCd \u2022\u2022 \n\nII \nII \nII \nII \nI \n\n*\"J>--........,'t--t--t-<.,fJ \n\n, .. , \n\n1 \nI \nI \nI \n\nI \nI \n\n724 \n\nO. INDNERI. J.  KRAMER. C. KOCH \n\n-\n\n-\n\n-\n\n-\n\n- -Ir -\n\n-\n\n-\n\n-\n\n-\n\n-\n\n-\n\n- - - - - - - - - - - - - - - - - - - ______ 1 ______ - - - -\n\nFigure  4:  Circuit schematics of the smoothing,  zero-detection  and  winner-take-all \nblocks  respectively. \n\n\"  .................. -r-~...,..... ........ ......-........ -.-................. -r-.,.......,.--.-, \n\" \n.B \n\n.0, \n\n.0. \n\n.0' \n\n'O\u00b7.~~'~~~~7~.~.~'.~'~,~I2~\"~'~.~'.~\"~'~\"~.~,,~~~'~' 22 \n\nlk1I P9ton \n\n\"!-, ~,  ~,~,~. ~'~'''''7-' ~,~,.\"\"\"~' ~,,~,,~,,~,\"\". ~,.\"\"\",,:-',~, .,..\" ~'O~\"~22' \n\n\"\"\"\"\"'''''' \n\n(a) \n\n(b) \n\nFigure 5:  Zero  crossings computed as difference between smoothed positive currents \nand  smoothed  negative  currents:  (a)  for  expanding  stimuli;  (b)  for  contracting \nstimuli.  The  \"zero\"  axis  is  shifted  due to is a  systematic offset  of 80  nA. \n\n4  Time-to-contact \n\nThe  time-to-contact  can  be  computed  by  exploiting  qualitative  properties  of the \noptical flow  field  such  as  expansion or contraction  [Poggio  et  al.,  1991].  The diver(cid:173)\ngence  theorem,  or  Gauss  theorem,  as  applied  to  a  plane,  shows  that  the  integral \nover  a  surface  patch of the  divergence  of a  vector  field  is  equal to the line  integral \nalong  the  patch  boundary  of the  component  of the  field  normal  to  the  boundary. \nSince  a  camera  approaching  a  rigid  object  sees  a  linear  velocity  field,  where  the \nvelocity  vectors  are  proportional to their distance  from the focus-of-expansion,  the \ndivergence  is  constant  over  the image plane.  By  integrating the  radial component \nof the optical flow  field  along the circumference  of a  circle,  the time-to-contact can \nthus be estimated, independently of the position of the focus-of-expansion. \n\nWe  implemented this  algorithm with  an  analog integrated  circuit,  where  an  array \nof twelve  motion sensing  elements  is  arranged  on  a  circle,  such  that  each  element \nmeasures velocity radially.  According  to the Gauss theorem, the time-to-contact is \n\n\fVLSI Architectures for  Computation of Heading Direction  and Time-to-contact \n\n725 \n\nthen  approximated by \n\nT= \n\nN\u00b7R \nN \n\n' \n\n2:k=l Vk \n\n(1) \n\nwhere N denotes the number of elements, R the radius of the circle, and Vk  the radial \nvelocity  components  at the  locations of the  elements.  For each  clement,  temporal \naliasing  is  prevented  by  comparing  the  output  voltages  of the  two  directions  of \nmotion and setting the lower one, corresponding to the null direction,  to zero.  The \noutput  voltages  are  then  used  to  control  subthreshold  transistor  currents.  Since \nthese voltages are logarithmically dependent on velocity, the transistor currents are \nproportional  to  the  measured  velocities.  The  sum  of the  velocity  components  is \nthus calculated by  aggregating the currents from  all  elements on two  lines,  one for \noutward  motion and one  for  inward motion, and  taking  the  difference  of the  total \ncurrents.  The  resulting  bi-directional output  current  is  an inverse  function  of the \nsigned  time-to-contact. \n\n;: \n1 o~--------~----~~ \nI \n\n-, \n\n-0 25 \n\nOOS \n\n01 \n\n015 \n\n02 \n\n025 \n\nTime-1c>ContKt (sec) \n\nFigure  6:  Output  current  of the  time-to-contact sensor  as  a  function  of simulated \ntime-to-contact under incandescent  room illumination.  The theoretical  fit  predicts \nan inverse  relationship. \n\nThe  circuit  has  been  implemented  on  a  chip  with  a  size  of (2.2mm)2  using  2  pm \ntechnology.  The photo diodes of the motion sensing  elements are  arranged on two \nconcentric circles with radii of 400 pm and 600 pm respectively.  In order to simulate \nan approaching or  withdrawing object,  a  high-contrast  spiral stimulus was  printed \nonto a  rotating disk.  Its image was  projected onto the chip with a  microscope lens \nunder  incandescent  room illumination.  The focus-of-expansion  was  approximately \ncentered  with  respect  to  the  photo  diode  circles.  The  averaged  output  current  is \nshown  as  a  function  of simulated  time-to-contact  with  a  theoretical  fit  in  Fig.  6. \nThe expected  inverse relationship is qualitatively observed  and the sign (expansion \nor contraction)  is  robustly  encoded.  However,  the deviation  of the output  current \nfrom its average can be substantial:  Since the output voltage of each motion sensing \nelement  decays  slowly  due  to  leak  currents  and  since  the spiral stimulus causes  a \nserial  update  of the  velocity  values  along  the  array,  a  step  change  in  the  output \ncurrent  is  observed  upon  each  update,  followed  by  a  slow  decay.  The  effect  is \naggravated, if the individual motion sensing elements measure significantly differing \nvelocities.  This is  generally the case,  because  the focus-of-expansion  is  usually not \ncentered  with  respect  to  the  sensor  and  because  of  inaccuracies  in  the  velocity \nmeasurements  due  to  circuit  offsets,  noise,  and  the  aperture  problem  [Verri  et  al., \n1992].  The integrative property of the algorithm is thus highly desirable,  and more \nrobust  data would  be obtained from  an array with  more elements and stimuli with \nhigher edge  densities. \n\n\f726 \n\nG. INDIVERI. J.  KRAMER. C.  KOCH \n\n5  Conclusions \n\nWe  have developed  parallel architectures for  motion analysis that bypass  the prob(cid:173)\nlem of low  precision in analog VLSI  technology  by exploiting qualitative properties \nof the optical  flow.  The  correct  functionality  of the  devices  built,  at  least  from  a \nqualitative point of view,  have confirmed  the validity of the  approach  followed  and \ninduced  us  to continue  this  line of research .  We  are  now  in  the process  of design(cid:173)\ning  more  accurate  circuits  that implement the operators  used  in  the  architectures \nproposed. \n\nAcknowledgments \n\nThis  work  was  supported  by  grants from  ONR,  ERe and  Daimler-Benz  AG.  The \nvelocity sensor  was  developed  in  collaboration with R . Sarpeshkar.  The chips  were \nfabricated through  the MOSIS  VLSI  Fabrication Service. \n\nReferences \n\n[Barron  et al.,  1994]  J.1. Barron,  D.J. Fleet,  and S.S.  Beauchemin.  Performance of optical \n\nflow  techniques.  International Journal on  Computer  Vision, 12(1):43-77,  1994. \n\n[Boahen  and Andreou,  1992]  K.A. Boahen  and A.G. Andreou.  A contrast sensitive silicon \n\nretina with  reciprocal  synapses.  In  NIPS91  Proceedings. IEEE,  1992. \n\n[Delbriick,  1991]  T. Delbriick.  \"Bump\"  circuits for  computing similarity  and dissimilarity \n\nof analog  voltages.  In  Proc.  IJCNN,  pages  1-475-479,  June  1991. \n\n[Kramer  et al.,  1995]  J.  Kramer,  R.  Sarpeshkar,  and  C.  Koch.  An  analog  VLSI  velocity \nsensor.  In  Proc.  Int.  Symp.  Circuit  and  Systems  ISCAS  '95,  pages  413-416,  Seattle, \nWA,  May  1995. \n\n[Mead,  1989]  C.A.  Mead.  Analog  VLSI and Neural  Systems.  Addison-Wesley,  Reading, \n\n1989. \n\n[Morris  et al.,  1995]  T .G.  Morris,  D.M. Wilson,  and S.P.  DeWeerth.  Analog VLSI circuits \nIn  Conference  for  Advanced Research  in  VLSI-Chapel \n\nfor  manufacturing  inspection. \nHill,  North  Carolina,  March  1995. \n\n[Poggio  et al.,  1991]  T.  Poggio,  A.  Verri,  and  V.  Torre.  Green  theorems  and  qualitative \nproperties  of the optical flow.  Technical  report,  MIT,  1991.  Internal  Lab.  Memo  1289. \n[Verri  et al.,  1992]  A.  Verri,  M.  Straforini,  and  V. Torre.  Computational  aspects of mo(cid:173)\ntion  perception  in  natural  and  artificial  vision  systems.  Phil.  Trans.  R.  Soc.  Lond.  B, \n337:429-443,  1992. \n\n\fSPEECH AND SIGNAL PROCESSING \n\nPART VI \n\n\f\f", "award": [], "sourceid": 1125, "authors": [{"given_name": "Giacomo", "family_name": "Indiveri", "institution": null}, {"given_name": "J\u00f6rg", "family_name": "Kramer", "institution": null}, {"given_name": "Christof", "family_name": "Koch", "institution": null}]}