{"title": "Dynamics of Attention as Near Saddle-Node Bifurcation Behavior", "book": "Advances in Neural Information Processing Systems", "page_first": 38, "page_last": 44, "abstract": null, "full_text": "Dynamics of Attention as  Near \n\nSaddle-Node Bifurcation Behavior \n\nHiroyuki Nakahara\" \nGeneral Systems Studies \n\nU ni versi ty of Tokyo \n\n3-8-1  Komaba,  Meguro \n\nTokyo  153,  Japan \n\nnakahara@vermeer.c.u-tokyo.ac.jp \n\nKenji Doya \n\nATR Human Information Processing \n\nResearch  Laboratories \n\n2-2  Hikaridai, Seika,  Soraku \n\nKyoto  619-02,  Japan \ndoya@hip.atr.co.jp \n\nAbstract \n\nIn  consideration  of attention  as  a  means  for  goal-directed  behav(cid:173)\nior in non-stationary environments,  we  argue that the dynamics of \nattention  should  satisfy  two  opposing  demands:  long-term  main(cid:173)\ntenance  and  quick  transition.  These  two  characteristics  are  con(cid:173)\ntradictory  within  the linear domain.  We  propose  the near saddle(cid:173)\nnode  bifurcation behavior of a  sigmoidal unit with self-connection \nas  a  candidate of dynamical mechanism that satisfies both of these \ndemands.  We  further  show  in  simulations  of  the  'bug-eat-food' \ntasks  that  the  near  saddle-node  bifurcation  behavior  of recurrent \nnetworks  can  emerge  as  a functional  property  for  survival in  non(cid:173)\nstationary environments. \n\n1 \n\nINTRODUCTION \n\nMost studies of attention have focused  on  the selection  process of incoming sensory \ncues  (Posner et  al.,  1980;  Koch  et  al.,  1985;  Desimone  et  al.,  1995).  Emphasis was \nplaced on the phenomena of causing different  percepts for  the same sensory stimuli. \nHowever,  the selection  of sensory  input  itself is  not  the final  goal  of attention.  We \nconsider  attention as a  means for  goal-directed behavior and survival of the animal. \nIn  this  view,  dynamical  properties  of attention  are  crucial.  While  attention  has \nto  be  maintained  long  enough  to  enable  robust  response  to  sensory  input,  it  also \nhas  to  be  shifted  quickly  to  a  novel  cue  that  is  potentially  important.  Long-term \nmaintenance and quick transition  are  critical  requirements  for  attention  dynamics. \n\n\u00b7currently  at  Dept.  of  Cognitive  Science  and  Institute  for  Neural  Computation, \n\nU.  C.  San  Diego,  La  Jolla CA  92093-0515.  hnakahar@cogsci.ucsd.edu \n\n\fDynamics of Attention as  Near Saddle-node Bifurcation Behavior \n\n39 \n\nWe  investigate  a  possible neural  mechanism that  enables  those  dynamical  charac(cid:173)\nteristics  of attention. \n\nFirst, we analyze the dynamics of a network of sigmoidal units with self-connections. \nWe  show  that  both  long-term  maintenance  and  quick  transition  can  be  achieved \nwhen  the system parameters are  near  a  \"saddle-node bifurcation\"  point .  Then,  we \ntest  if such  a  dynamical  mechanism  can  actually  be  helpful  for  an  autonomously \nbehaving  agent  in  simulations of a  'bug-eat-food'  task.  The  result  indicates  that \nnear  saddle-node  bifurcation  behavior  can  emerge  in  the  course  of evolution  for \nsurvival in non-stationary environments. \n\n2  NEAR SADDLE-NODE BIFURCATION  BEHAVIOR \n\nWhen  a  pulse-like  input  is  given  to  a  linear  system,  the  rising  and  falling  phases \nof the  response  have  the same  time  constants.  This means  that long-term mainte(cid:173)\nnance  and  quick  transition cannot  be simultaneously achieved  by  linear  dynamics. \nTherefore,  it  is  essential  to  consider  a  nonlinear  dynamical  mechanism  to  achieve \nthese  two  demands. \n\n2.1  DYNAMICS  OF  A  SELF-RECURRENT UNIT \n\nFirst,  we  consider  the dynamics of a  single sigmoidal unit with  the self-connection \nweight  a and the  bias  b. \n\ny(t + 1) \nF(x) \n\nF(ay(t) + b) , \n\n1 \n\n1 + exp( -x)' \n\n(1) \n\n(2) \n\nThe parameters (a, b)  determine  the qualitative behavior of the system such  as the \nnumber  of fixed  points  and  their  stabilities.  As  we  change  the  parameters ,  the \nqualitative  behavior  of the  system  may  suddenly  change.  This  is  referred  to  as \n\"bifurcation\"  (Guckenheimer,  et  al.,  1983).  A  typical  example  is  a  \"saddle-node \nbifurcation\"  in  which  a  pair of fixed  points,  one stable  and one  unstable,  emerges. \nIn our system,  this occurs when  the state transition curve  y(t + 1)  =  F(ay(t) + b)  is \ntangent  to y(t + 1)  =  y(t).  Let  y*  be this point of tangency.  We  have  the following \ncondi tion for  saddle-node bifurcation. \n\nF(ay* + b) \ndF(ay + b) I \n\ndy \n\ny=y. \n\ny* \n\n1 \n\nThese equations can  be solved,  by  noting F'(x) = F(x)(l- F(x)),  as \n\na \n\n1 \n\ny* (1 - y*) \n\n(3) \n\n( 4) \n\n(5) \n\n1 \n\nb  = \n\nF-1(y*) - ay*  =  F-l(y*) - - -\nI- y* \n\n(6) \nBy  changing  the fixed  point  value  y*  between  a and  1, we  can  plot  a  curve  in  the \nparameter space (a, b)  on which saddle-node bifurcation occurs,  as shown in  Figure \n1  (left).  A  pair  of a  saddle  point  and  a  stable  fixed  point  emerges  or  disappears \nwhen  the  parameters  pass  across  the  cusp  like  curve  (cases  2  and  4) .  The system \nhas  only one stable fixed  point  when  the  parameters  are outside  the  cusp  (case  1) \nand three  fixed  points inside the cusp  (case  3). \n\n\f40 \n\nH.NAKAHARA,K.DOYA \n\nb  Bifurcat ion  Diagr am \n\ny ( t+ l ) \n\nCASE  1 \n\nC. 8f::.x.d  Pt \u2022. , ~' 1 \n06, \n\n\" ,  \n\n., \n\n-'0 \n\n- 15 \n\n- lO \n\n041 \n\n\" \n\nO~/  ../ \n\nH  0.20. 40.60 8  ly(tJ \n\n\u00b7\" CASE 3 \nY'~tL\n' \n.\no. a: \n,,/ \n't1x.d  p t \n\"'  3 \n, \n(I  6 \nOJ'\no \n\n'' \n, \n,/ \n\n.,\n' \n\n'0  20  40  60  8  1 y( t ) \n\ny(t +1 J \n\nCASE  2 \n\n\" \n~  8:tixed  pts,' \n0  61 \n0  \"i \n0'1/ \n\n,,' \n\n\" \n\n\"\"0~1O:\".,,,,0 ';;;-0;;-;8 lY lt ) \n\n. \n\n0  B \n\nY',t.\" CASE \u2022 \n. \n. \n,,' \n\n0 \" \n\n0  6 \n\n\" \n\n' \n\nf lxed  p ts  \" .  2 \n\n0 2: \n\n' \n0  2('  4~  60  8:  1 y (tJ \n\nFigure  1:  Bifurcation  Diagram  of a  Self-Recurrent  Unit.  Left :  the  curve  in  the \nparameter space  (a , b)  on  which saddle-node  bifurcation is  seen.  Right :  state tran(cid:173)\nsition diagrams for  four  different  cases. \n\ny ( t \u00b7 l) \n\no'~=Lil\n\nl. 1111  b  = - 7. 9 \n\n0.6 \n\n0. 4 \n0 .2 \na \n\nI \n\n\" \n\nI \n... \" \n\n0.20. 4 0 . 60. 81 \n\ny et) \n\nY1d(t:l)ll.ll11  b  = ~9 \no. \nO. \n\nI \nI \n\nO. \nO. \n\no \n\n\" \n\nI \n\nI , \n\no . 20 . 4 0  60. 8  1 \n\nyet) \n\no .~ o. \n\nyet) \n\ndL \n\no. & \n0 . 41 \n0.21 \no \n\ny et ) \n\no. \no. \n\no \n\n5  10  15  2 (f i me (t) \n\n5  1 0 \n\nl S  2a:'i rne l t ) \n\nFigure  2:  Temporal  Responses  of  Self-Recurrent  Units.  Left :  near  saddle-node \nbifurcation.  Right : far  from bifurcation. \n\nAn interesting behavior can  be seen  when the parameters are just outside the cusp, \nas  shown  in  Figure  2  (left) .  The system  has  only  one  fixed  point  near  Y = 0,  but \nonce the  unit is  activated (y  ~ 1), it stays  \"on\"  for  many time steps  and then  goes \nback  to the fixed  point  quickly.  Such  a  mechanism may be  useful  in  satisfying the \nrequirements of attention dynamics:  long-term maintenance and quick  transition. \n\n2.2  NETWORK  OF  SELF-RECURRENT  UNITS \n\nNext,  we  consider  the dynamics of a  network of the  above self-recurrent  units. \n\nYi(t + 1) =  F[aYi(t) + b + L CijYj(t) + diUi(t)], \n\nj,jti \n\n(7) \n\nwhere a is the self connection weight , b is the bias, Cij  is the cross connection weight, \nand di  is the input connection  weight , and Ui(t)  is  the external input.  The effect  of \nlateral and external inputs is  equivalent to the change  in the bias, which slides  the \nsigmoid curve  horizontally without changing the slope. \nFor  example,  one  parameter  set  of the  bifurcation  at  y*  = 0.9  is  a  = 11.11  and \nb  ~ -7.80.  Let  b = -7.90  so  that  the  unit  has  a  near  saddle-node  bifurcation \nbehavior  when  there  is  no  lateral or  external  inputs.  For  a  fixed  a =  11.11,  as  we \nincrease  b, the qualitative behavior of the system appears as case  3 in  Figure 1, and \n\n\fDynamics of Attention as  Near Saddle-node Bifurcation Behavior \n\n41 \n\nSensory Inputs \n\n, \n\n, \n\n'olr lood \n\n\" \n\n- '--,~,- \\,  :/~-- - . \n\n'on:'oocl  ~~ ....... \n... \n\nr-.~ \"==:-:\"~-\n~~  ....\n-\n\nInpol.  IJrI  .111'2 \n\n..... \n\nCreature \n\n... \n\nActions \n\nNetwork Structure \n\nCreature \n\nFigure  3:  A  Creature's  Sensory  Inputs(Left),  Motor  System(Center)  and  Network \nArchitecture(Right) \n\nthen,  it  changes  again  at  b::::::  -3.31,  where  the  fixed  point  at  Y  =  0.1,  or  another \nbifurcation point , appears as  case  4 in Figure  L  Therefore, ifthe input sum is  large \nenough,  i.e.  L j ,j;Ci CijYj  + diuj  >  -3.31- (-7.90)  ::::::  4.59,  the  lower  fixed  point \nat  Y = 0.1  disappears  and  the state jumps up  to  the  upper  fixed  point  near  Y = 1, \nquickly  turning  the  unit  \"on\".  If the  lateral  connections  are  set  properly,  this  can \nin  turn  suppress  the  activation of other  units.  Once  the  external  input  goes  away, \nas  we  see  in  Figure  2 (left),  the state stays  \"on\"  for  a  long  time until  it  returns  to \nthe fixed  point near  Y =  O. \n\n3  EVOLUTION  OF  NEAR BIFURCATION  DYNAMICS \n\nIn  the  above  section,  we  have  theoretically  shown  the  potential  usefulness  of near \nsaddle-node bifurcation behavior for satisfying demands for attention dynamics.  We \nfurther  hypothesize that such behavior is  indeed useful in  animal behaviors and can \nbe found  in the  course  of learning  and evolution  of the  neural system. \n\nTo  test  our  hypothesis,  we  simulated  a  'bug-eat-food'  task .  Our  purpose  in  t.his \nsimulation  was  to  see  whether  the  attention  dynamics  discussed  in  the  previous \nsection  would help  obtain better performance in  a non-stationary environment.  Vve \nused  evolutionary  programming (Fogel  et  aI,  1990)  to  optimize the performance  of \nrecurrent  networks  and feedforward  networks. \n\n3.1  THE  BUG  AND  THE  WORLD \n\nIn  our simulation, a simple creature traveled  around a  non-stationary environment. \nIn  the  world,  there  were  a  certain  number of food  items.  Each  item was  fixed  at  a \ncertain  place  in  the  world  but  appeared  or  disappeared  in  a  stochastic  fashion,  as \ndetermined  by  a  two-state  Markov  system.  In  order  to survive,  A  creature  looked \nfor  food  by  traveling  the  world .  The  amount  of food  a  creature  found  in  a  certain \ntime  period  was  the  measure  of its  performance. \nA  creature  had five  sensory  inputs,  each  of which  detected  food  in  the  sector  of 45 \ndegrees  (Figure  3,  right).  Its  output  level  was  given  by  L J'  .l..,  where  Tj  ,\"vas  the \ndistance  to the j-th food  item within the sector.  Note  that the format  of the input \ncontained information about  distance  and also  that the creature  could only receive \nthe  amount  of the input  but  could  not  distinguish each food  from  others. \n\nr J \n\nFor  the  sake  of simplicity,  we  assumed  that  the  creature  lived  in  a  grid-like world . \nOn each  time step,  it took one of three motor commands:  L:  turn left  (45  degrees), \n\n\f42 \n\nH.  NAKAHARA,  K.  DOYA \n\nDensity of Food \nMarkov Transition  Matrix \nof each food \nRandom Walk \nNearest  Visible \nFeedForward \nRecurrent \nNearest  Visible/Invisible \n\n0.05 \n\n0.10 \n\n.5  .5 \n.5  .5 \n7.0 \n42.7 \n58.6 \n65.7 \n97.7 \n\n.8  .8 \n.2  .2 \n6.9 \n18.6 \n37.3 \n43.6 \n97.1 \n\n.5  .5 \n.5  .5 \n13.8 \n65.3 \n84.8 \n94.0 \n129.1 \n\n.8  .8 \n.2  .2 \n13.9 \n32.4 \n60.0 \n66.1 \n128.8 \n\nTable  1:  Performances  of the Recurrent  Network  and  Other Strategies. \n\nC:  step  forward,  and  R:  turn  right  (Figure  3,  center).  Simulations  were  run  with \ndifferent  Markov  transition  matrices  of food  appearance  and  with  different  food \ndensities.  A  creature  got  the food  when  it reached  the food,  whether  it  was  visible \nor invisible.  When a creature ate a food item, a new food item was placed randomly. \nThe size  of the  world  was  10x10 and  both ends  were  connected  as  a  torus. \n\nA  creature  was  composed  of two  layers:  visual  layer  and  motor  layer  (Figure  3, \nleft).  There  were  five  units 1  in  visual  layer,  one  for  each  sensory  input,  and  their \ndynamics  were  given  by  Equation  (7).  The  self-connection  a,  the  bias  b and  the \ninput weight  di  were  the same for  all  units.  There  were  three  units in  motor layer, \neach  coding  one of three  motor commands, and  their state  was  given  by \n\nek  + L: fkiYi(t), \nexp(xk(t)) \nL:/ exp(x/(t)) ' \n\n(8) \n\n(9) \n\nwhere  ek  was the  bias and  fki  was the feedforward  connection  weight. 2  One of the \nthree  motor  commands  (L,C,R)  was  chosen  stochastically  with  the  probability  Pk \n(k=L,C,R).  The  activation  pattern  in  visual  layer  was  shifted  when  the  creature \nmade a  turn,  which should give  proper mapping between  the sensory input  and the \nworking  memory. \n\n3.2  EVOLUTIONARY  PROGRAMMING \n\nEach  recurrent  network  was  characterized  by  the  parameters  (a,b,Cij,di,ek,lkd, \nsome  of  which  were  symmetrically  shared,  e.g.  C12  =  C21.  For  comparison,  we \nalso  tested  feedforward  networks  where  recurrent  connections  were  removed,  i.e. \na = Cij  = O. \nA  population of 60  creatures  was  tested  on each generation.  The initial population \nwas  generated  with  random parameters.  Each  of the  top  twenty  scoring  creatures \nproduced  three  offspring;  one  identical  copy  of the  parameters  of the  parent's  and \ntwo copies of these parameters with a Gaussian fluctuation.  In  this paper, we  report \nthe  result  after  60  generations. \n\n3.3  PERFORMANCE \n\nthe later convenience \n\n1 We  denote  each  unit  in  visual  layer  by  Ul, U2, U3, U4, Us  from  the  left  to  the  right  for \n2In  this simulation  reported  here,  we  set  ek  = O. \n\n\fDynamics of Attention as  Near Saddle-node Bifurcation Behavior \n\n43 \n\n-, \n\n-, , \n-, \n\n- 7 \n\n, \n\n- 10 \n\n- 12 . 5 \n\n-,  , . . \" \n\n, \n\n.... : .... \n\n-,  , \n\n_7 \n\n-L25 \n\na \n\nb \n\n\"Transition matrix = (  :~  .5  ) \n.5 \n\nbTransition  matrix =  ( \n\n:~ \n\n:~ ) \n\nFigure 4: The Convergence of the Parameter of (a , b)  by Evolutionary Programming \nPlotted  in  the  Bifurcation  Diagram.  The  food  density  is  0.10  in  both  examples \nabove. \n\nTable  1  shows  the  average  of food  found  after  60  generations.  As  a  reference  of \nperformance  level,  we  also  measured  the  performances  of three  other  simple  algo(cid:173)\nrithms:  1)  random walk : one of the three  motor commands is  taken randomly with \nequal  probability.  2)  nearest  visible:  move  toward  the  nearest  food  visible  at  the \ntime  within  the  creature's  field  of view  of (U2, U3, U4).  3)  nearest  visible/invisible: \nmove toward the nearest food  within the view of (U2, U3, U4)  no matter if it is  visible \nor not,  which gives  an upper  bound of performance. \n\nThe  performance  of recurrent  network  is  better  than  that  of feedforward  network \nand  'nearest visible'.  This suggests that the  ability of recurrent  network  to remem(cid:173)\nber  the past is  advantageous. \n\nThe  performance  of feedforward  network  is  better  than  that  of  'nearest  visible '. \nOne reason is  that feedforward  network could cover a broader area to receive  inputs \nthan 'nearest visible'.  In addition, two factors,  the average time in which  a creature \nreaches  the food  and the  average  time in  which  the food  disappears,  may influence \nthe performance of feedforward  network  and 'nearest visible'.  Feedforward  network \ncould  optimize  its output  to  adapt  two  factors  with  its  broader  view  in  evolution \nwhile  'nearest  visible'  did not have such  adaptability. \n\nIt should be noted that both of 'nearest visible/invisible' and 'nearest visible' explic(cid:173)\nitly assumed the higher-order sensory processing:  distinguishing each food item from \nthe others and measuring the distance between each food and its body.  Since its per(cid:173)\nformance  is  so  different  regardless  of its higher-order sensory  processing, it  implies \nthe importance of remembering the past.  We  can regard recurrent  network  as  com(cid:173)\npromising  two  characteristics,  remembering  the  past  as  'nearest  visible/invisible' \ndid  and  optimizing the  sensitivity  as  feedforward  network  did ,  although  recurrent \nnetwork  did not  have  a  perfect  memory  as 'nearest  visible/invisible' . \n\n3.4  CONVERGENCE TO  NEAR-BIFURCATION  REGIME \n\nWe  plotted the  histogram of the performance  in  each  generation and the history  of \nthe  performance  of a  top-scoring  creature  over  generations.  Though  they  are  not \nshown  here,  the  performance  was  almost optimal after 60  generations. \n\nFigure  4  shows  that  two  examples  of a  graph  in  which  we  plotted  the  parameter \n\n\f44 \n\nH.  NAKAHARA,  K.  DOYA \n\nset  (a , b)  of top  twenty scoring  creatures  in  the  60th  generation  in  the  bifurcation \ndiagram.  In the left graph,  we  can  see  the parameter set  has converged  to a  regime \nthat gives  a  near saddle-node bifurcation behavior.  On the other hand,  in  the right \ngraph,  the  parameter  set  has  converged  into  the  inside  of cusp.  It is  interesting \nto  note  that  the  area  inside  of the  cusp  gives  bistable  dynamics.  Hence,  if the \ninput  is  higher  than  a  repelling  point,  it  goes  up  and  if the  input  is  lower, it goes \ndown .  The  reason  of the  convergence  to  that  area is  because  of the  difference  of \nthe world setting, that is,  a  Markov  transition  matrix.  Since food  would  disappear \nmore  quickly  and  stay  invisible  longer  in  the  setting  of the  right  graph,  it  should \nbe  beneficial  for  a  creature  to  remember  the  direction  of higher  inputs  longer.  In \nmost of cases  reported  in Table  1, we  obtained the  convergence  into our  predicted \nregime  and/or the inside of the cusp. \n\n4  DISCUSSION \n\nNear  saddle-node  bifurcation  behavior  can  have  the  long-term  maintenance  and \nquick  transition,  which  characterize  attention  dynamics.  A  recurrent  network \nhas  better  performance  than  memoryless  systems  for  tasks  in  our  simulated  non(cid:173)\nstationary  environment.  Clearly,  near  saddle-node  bifurcation  behavior  helped  a \ncreature's  survival  and  in  fact,  creatures  actually evolved  to  our  expected  param(cid:173)\neter  regime .  However,  we  also  obtained  the  convergence  into  another  unexpected \nregime which gives bistable dynamics .  How  the bistable dynamics are used  remains \nto  be investigated. \n\nAcknowledgments \n\nH.N . is  grateful to Ed Hutchins for  his generous support,  to John Batali and David \nFogel  for  their  advice  on  the  implementation of evolutionary  programming and  to \nDavid Rogers for  his comments on the manuscript of this  paper. \n\nReferences \n\nR.  Desimone,  E.  K.  Miller, L.  Chelazzi,  &  A.  Lueschow.  (1995)  Multiple  Memory \nSystems in the Visual Cortex.  In  M. Gazzaniga (ed .),  The  Cognitive  Neurosciences, \n475-486.  MIT  Press. \nD.  B.  Fogel,  L. J.  Fogel,  &  V.  W . Porto.  (1990)  Evolving Neural  Networks.  Biolog(cid:173)\nical cybernetics 63:487-493. \nJ.  Guckenheimer  &  P.  Homes.  (1983)  Nonlinear  Oscillations,  Dynamical  Systems, \nand  Bifurcation  of Vector  Fields \n\nC.  Koch &  S.  Ullman .  (1985)  Shifts in selective visual attention:towards the under(cid:173)\nlying neural circuitry.  Human  Neurobiology 4:219-227 . \nM.  Posner, C .. R  .R. Snyder, &  B. J.  Davidson.  (1980)  Attention and the detection \nof signals.  Journal  of Experimental Psychology:  General 109:160-174 \n\n\f", "award": [], "sourceid": 1031, "authors": [{"given_name": "Hiroyuki", "family_name": "Nakahara", "institution": null}, {"given_name": "Kenji", "family_name": "Doya", "institution": null}]}