{"title": "Competence Acquisition in an Autonomous Mobile Robot using Hardware Neural Techniques", "book": "Advances in Neural Information Processing Systems", "page_first": 1031, "page_last": 1037, "abstract": null, "full_text": "Competence Acquisition  in an \n\nAutonomous  Mobile Robot  using \n\nHardware Neural Techniques. \n\nGeoff Jackson and Alan F.  Murray \n\nDepartment of Electrical  Engineering \n\ngbj@ee.ed.ac.uk,afm@ee.ed.ac.uk \n\nEdinburgh  University \nEdinburgh,  ER9  3JL \n\nScotland,  UK \n\nAbstract \n\nIn  this  paper  we  examine  the  practical  use  of  hardware  neural \nnetworks  in  an  autonomous  mobile  robot.  We  have  developed  a \nhardware  neural  system  based  around  a  custom  VLSI  chip,  EP(cid:173)\nSILON  III,  designed  specifically  for  embedded  hardware  neural \napplications.  We  present  here  a  demonstration  application  of an \nautonomous mobile robot that highlights the flexibility of this sys(cid:173)\ntem.  This robot gains basic mobility competence in very few  train(cid:173)\ning epochs  using  an  \"instinct-rule\"  training methodology. \n\n1 \n\nINTRODUCTION \n\nThough neural networks have been shown as an effective solution for a diverse range \nof real-world problems, applications and especially hardware implementations have \nbeen  few  and  slow  to  emerge.  For  example in  the  DARPA  neural  networks  study \nof 1988;  of the  77  neural  network  applications investigated  only  4  had  resulted  in \nfield  tested  systems  [Widrow,  1988].  Furthermore,  none  of these  used  dedicated \nneural network  hardware.  It is  our  view  that this lack  of tangible successes  can  be \nsummarised by  the following points: \n\n\u2022  Most  neural  applications  will  be  served  optimally  by  fast,  generic  digital \n\ncomputers . \n\n\u2022  Dedicated digital neural accelerators have a limited lifetime as  \"the fastest\" , \n\nas standard  computers develop  so  rapidly. \n\nlEdinburgh  Pulse  Stream Implemenation  of a  Learning  Oriented  Network. \n\n\f1032 \n\nG. JACKSON, A.  F. MURRAY \n\n\u2022  Analog neural VLSI is a niche technology, optimally applied at the interface \n\nbetween  the  real  world  and higher-level  digital processing. \n\nThis  attitude  has some profound  implications with  respect  to  the size,  nature  and \nconstraints we  place on new hardware neural designs.  After several years of research \ninto  hardware  neural  network  implementation,  we  have  now  concentrated  on  the \nareas in which analog neural network technology has an  \"edge\"  over well established \ndigital technology. \n\nWithin  the  pulse  stream  neural  network  research  at  the  University  of Edinburgh, \nthe  EPSILON  chip's areas of strength  can  be summarised as: \n\n\u2022  Analog or  digital inputs,  digital outputs. \n\u2022 \n\nScaleable and  cascadeable  design. \n\n\u2022  Modest  size. \n\u2022  Compact, low  power. \n\nThis  list  points  naturally  and  strongly  to  problems  on  the  boundary  of the  real, \nanalog world and digital processing, such as pre-processing/interpretation of analog \nsensor data.  Here a modest neural network can act as an intelligent analog-to-digital \nconverter  presenting  preprocessed  information  to  its  host.  We  are  now  engaged \nin  a  two  pronged  approach,  whereby  development  of  technology  to  improve  the \nperformance  of pulse  stream  neural  network  chips  is  occurring  concurrently  with \na  search  and  development of applications to which  this  technology  can  be  applied. \nThe key  requirements of this technological  development are  that devices  must: \n\n\u2022  Work directly  with analog signals. \n\u2022  Provide a  moderate size  network. \n\u2022  Have  the  potential for  a fully  integrated solution. \n\nIn  working  with  the  above  constraints  and  goals  we  have  developed  a  new  chip, \nEPSILON  II,  and  a  bus  based  processor  card  incorporating  it .  It is  our  aim  to \nuse  this system  to develop  applications.  As  our first  demonstration  the  EPSILON \nprocessor  card has been  mounted on an autonomous mobile robot.  In this case  the \nnetwork  utilises  a  mixture of analog and digital sensor  information and  performs a \nmapping between  input/sensor space,  a  mixture of analog and  digital signals,  and \noutput  motor control. \n\n2  THE EPSILON II CHIP \n\nThe EPSILON  II chip has been  designed around the requirements of an application \nbased system.  It follows on from an earlier generation of pulse stream neural network \nchip,  the  EPSILON  chip  [Murray,  1992]. \n\nThe EPSILON II chip represents neural states as a pulse encoded signal.  These pulse \nencoded signals have  digital signal  levels which  make them highly immune to noise \nand  ideal  for  inter  and intra-chip  communication, facilitating efficient  cascading of \nchips  to form larger systems.  The EPSILON  II chip  can  take as inputs either  pulse \nencoded  signals  or  analog  voltage  levels,  thus  facilitating the  fusing  of analog  and \ndigital  data  in  one  system.  Internally  the  chip  is  analog  in  nature  allowing  the \nsynaptic  multiplication function  to  be  carried  out  in  compact  and  efficient  analog \ncells  [J ackson,  1994]. \nTable  1  shows  the  principal  specifications  of  the  EPSILON  II  chip.  The  EPSI(cid:173)\nLON  II  chip  is  based  around  a  32x32 synaptic  matrix allowing efficient  interfacing \nto  digital systems.  Several  features  of the  device  have  been  developed  specifically \nfor  applications based usage.  The first of these is a  programmable input mode.  This \n\n\fCompetence Acquisition  in  an  Autonomous  Mobile  Robot \n\n1033 \n\nTable  1:  EPSILON  II  Specifications \n\nEPSILON  II  Chip Specifications \n\n32 \nAnalog t PW or PF \nBit programmable \n32  pinned out \nPW or PF \n\nNo.  of state input pins \nInput modes \nInput mode programmability \nNo. of state outputs \nOutput modes \nDigital recovery  of analog liP  Yes  - PW encoded \nNo.  of Synapses \nAdditional  autobias synapses \nWeight storage \nProgrammable activity voltage  Yes \nDie size \n\n1024 \n4 per  output neuron \nDynamic \n\n6.9mm  x  7mm \n\nallows each of the network inputs to be programmed as either a  direct  analog input \nor a  digital pulse encoded  input.  We  believe  that this is  vital for  application based \nusage  where  it is  often  necessary  to fuse  real-world analog data with  historical or \ncontrol data generated digitally.  The second major feature is a pulse recovery mode. \nThis allows conversion of any analog input into a  digital value for  direct  use  by  the \nhost  system.  Both these  features  are  utilised in the robotics  application described \nin section  4 of this paper. \n\n3  EPSILON PROCESSOR CARD \n\nThe  need  to  embed  the  EPSILON  chip  in  a  processor  card  is  driven  by  several \nconsiderations.  FirstlYt  working  with  pulse  encoded  signals  requires  substantial \nprocessing  to  interface  directly  to  digital  systems.  If the  neural  processor  is  to \nbe  transparent  to  the  host  system  and  is  not  to  become  a  substantial  processing \noverhead t then  all  pulse  support  operations  must  be  carried  out  independently  of \nthe host system.  SecondlYt  to respond to further  chip level advances and allow rapid \nprototyping  of new  applications  as  they  emerge t  a  certain  amount of flexibility  is \nneeded  in the system.  It is  with these  points in mind that the design  of the flexible \nEPSILON  Processor  Card (EPC)  was  undertaken . \n\n3.1  DESIGN  SPECIFICATION \n\nThe EPC  has  been  designed  to meet the following specifications.  The card must : \n\n\u2022  Operate on a  conventional digital bus system. \n\u2022  Be  transparent  to  the  host  processor t  that  is  carry  out  all  the  necessary \n\npulse  encoding  and decoding. \n\n\u2022  Carry  out  the  refresh  operations  of  the  dynamic  weights  stored  on  the \n\nEPSILON  chip. \n\n\u2022  Generate  the  ramp waveforms necessary for  pulse  width  coding. \n\u2022  Support  the operation of multiple EPCts. \n\u2022  Allow  direct  input of analog signals. \n\nAs all data used and generated by the chip is effectively of 8-bit resolution t the STE \nbUSt  an industry standard 8-bit bUSt  was chosen for the bus system.  This is also cost \n\n\f1034 \n\nG. JACKSON, A.  F.  MURRAY \n\neffective  and  allows  the  use  of readily  available  support  cards  such  as  processors, \nDSP  cards  and  analog and  digital signal conditioning cards. \n\nTo allow the transparency of operation the card must perform a variety of functions . \nA  block  diagram indicating these functions  is  shown  in figure  1. \n\n\u00b7 \n................... . . . .......................... __ ........ . \n. \n\u00b7 \n. \n. \n\u00b7 \n\u00b7 \n. \n\u00b7 \n. \n\u00b7 \n. \n\n1--\"\"':---1  Pulse to Dig. Conv. \n\nFPGA \n\nDig. to  Pulse Cony. \n\nWeight refresh Ctrl. \n\nWeight RAM \n\nFigure  1:  EPSILON  Processor  Card \n\nA substantial amount of digital processing  is  required  by  the card,  especially in the \npulse  conversion  circuitry.  To  conform  to  the  Eurocard  standard  size  of the  STE \nspecification  an  FPGA device  is  used  to  \"absorb\"  most of the digital logic.  A  twin \nmother/daughter board design is also used  to isolate sensitive analog circuitry from \nthe  digital  logic.  The  use  of the  FPGA  makes  the  card  extremely  versatile  as  it \nis  now  easily  reconfigurable  to  adapt  to  specialist  application.  The  dotted  box  of \nfigure  1 shows  functions  implemented by  the  FPGA device.  An  on  board  EPROM \ncan  hold  multiple  FPGA  configurations  such  that  the  board  can  be  reconfigured \n\"on  the  fly\" .  All  EPSILON  support  functions ,  such  as  ramp  generation,  weight \nrefresh,  pulse conversion  and interface  control are  carried out on  the card.  Also the \nuse  of the  FPGA means that  new  ideas  are  easily  tested  as  all digital signal  paths \ngo  via  this  device.  Thus  a  card  of new  functionality  can  be  designed  without  the \nneed  to design  a  new  PCB. \n\n3.2  SPECIALIST BUSES \n\nThe  digital pulse  bus  is  buffered  out  under  control  of the  FPGA to the  neural  bus \nalong with two control signals.  Handshaking between EPC's is done over these lines \nto  allow  the  transfer  of pulse  stream  data  between  processors.  This  implies  that \nlarger networks  can  be  implemented with little or no  increase  in  computation time \nor overhead.  A separate analog bus is included  to bring analog inputs directly onto \nthe  chip. \n\n4  APPLICATIONS  DEVELOPMENT \n\nThe over-riding reason for  the development of the  EPC is to allow the easy develop(cid:173)\nment of hardware neural  network  applications.  We  have  already indicated  that  we \nbelieve  that  this  form  of neural  technology  will  find  its niche  where  its  advantages \nof direct  sensor  interface,  compactness  and  cost-effectiveness  are  of prime  import(cid:173)\nance.  As  a  good and intrinsically interesting example of this genre  of applications, \nwe  have  chosen  autonomous mobile robotic  control  as  a  first  test for  EPSILON  II. \nThe  object  of this  demonstrator  is  not  to  advance  the  state-of-the-art  in  robotics. \n\n\fCompetence Acquisition in  an  Autonomous  Mobile  Robot \n\n1035 \n\nRather it is  to demonstrate analog neural  VLSI  in  an appropriate and stimulating \ncontext. \n\n4.1 \n\n\"INSTINCT-RULE\"  ROBOT \n\nThe  \"instinct-rule\"  robotic control philosophy is  based on a  software-controlled ex(cid:173)\nemplarfrom the University's Department of Artificial Intelligence [Nehmzow, 1992]. \nThe  robot  incorporates  an  EPC  which  interfaces  all  the  analog sensor  signals  and \nprovides  the  programmable neural  link  between  sensor/input space  and the  motor \ndrive  actuators. \n\n~ -\"*,::::--,,, \n'\" o en \nffi  -..,...,\\--,,L \nen \n\na) Controller Architecture. \n\nb)  Instinct rule robot. \n\nFigure 2:  \"Instinct  Rule\"  Robot \n\nThe controller architecture is shown in figure  2.  The neural network implemented on \nthe EPC  is  the  plastic element that determines  the mapping between  sensory  data \nand  motor actions.  The majority of the  monitor section  is  currently  implemented \non  a  host  processor  and monitors the  performance of the  neural  network.  It does \nthis by  regularly evaluating a set of instinct rules.  These rules are simple behaviour \nbased  axioms.  For example, we  use  two rules  to promote simple obstacle  avoidance \ncompetence in  the  robot,  as  listed  in column one of table  2 \n\nTable 2:  Instinct  Rules \n\nSimple obstacle  avoidance. \n\nl.  Keep crash sensors inactive. \n2.  Move forward. \n\nWall following \n\nl.  Keep  crash sensors inactive. \n2.  Keep  side sensors active. \n3.  Move forward. \n\nIf an  instinct  rule  is  violated  the  drive  selector  then  chooses  the  next  strongest \noutput  (motor  action)  from  the  neural  network.  This  action  is  then  performed  to \nsee  if it  relieves  the  violation.  If it  does,  it  is  used  as  targets  to  train  the  neural \nnetwork .  If  it  does  not,  the  next  strongest  action  is  tried.  The  mechanism  to \naccomplish this will  be  described  in  more  detail  in section  4.2 . \n\nUsing this scheme the robot can be initialised with random weights (i.e. no mapping \nbetween  sensors  and motor control) and within a few  epochs  obtains basic obstacle \navoidance  competence. \n\nIt is  a  relatively  easy  matter  to  promote  more  complex  behaviour  with  the  ad(cid:173)\ndition  of other  rules.  For  example  to  achieve  a  wall  following  behaviour  a  third \n\n\f1036 \n\nG. JACKSON, A.  F. MURRAY \n\nrule  is  introduced  as  shown  in  column  two  of table  2.  Navigational  tasks  can  be \naccomplished  with  the  addition of a  \"maximise navigational signal\"  rule.  An \nexample of this  is  a  light  sensor  mounted  on  the  robot  producing  a  behaviour  to \nmove towards  a  light source.  Equally,  a  signal from  a  more  complex, higher  level, \nnavigational  system  could  be  used.  Thus  the  instinct  rule  controller  handles  ba(cid:173)\nsic  obstacle  avoidance  competence and motor/sensory interface  tasks leaving other \nresources  free  for  intensive  navigational tasks. \n\n4.2 \n\nINSTINCT  RULE EVALUATION  USING  SOMATIC  TENSION \n\nThe original instinct  rule  robot  used  binary sensor signals and  evaluated  perform(cid:173)\nance  of  alternative  actions  for  fixed,  and  progressively  longer,  periods  of  time \n[Nehmzow,  1992].  With the EPC interfacing directly to analog sensors an improved \nscheme  has  been  developed.  If we  sum  all  sensors  onto  a  neuron  with  fixed  and \nequal  weights  we  gain  a  measure of total sensory  activity.  Let  us  call  this  somatic \ntension  as  an  analogy  to  biological  signal  aggregation  on  the  soma.  If we  have \nan  instinct  violation  and  an  alternative  action  is  performed  we  can  monitor  this \nsomatic tension  to gauge the  performance of this action.  If tension  decreases  signi(cid:173)\nficantly  we  continue the action.  If it increases significantly we  choose  an alternative \naction.  If tension  remains  high  and roughly  the  same,  we  are  in  a  tight  situation, \nfor  example say  a  corner.  In  this  case  we  perform  actions for  progressively  longer \nperiods  continuing to monitor somatic tension for  a  drop. \n\n4.3  RESULTS  AND  DISCUSSION \n\nThe instinct rule robot has been constructed and its performance is comparable with \nsoftware-controlled predecessors.  Unfortunately direct comparisons are not possible \ndue  to unavailability of the original exemplars and differing physical characteristics \nof the  robots  themselves.  In  developing  the  application several  observations  were \nmade concerning  the  behaviour of the system that would  not have come to light in \na simulated environment. \n\nIn any system  including real  mechanics  and real  analog signals,  imperfections and \nnoise  are present.  For  example, in a  real robot  we  cannot guarantee that a  forward \nmotion directive  will  result  in  perfect  forward  motion due to inherent  asymmetries \nin  the  system.  The  instinct  rule  architecture  does  not  assume  a-priori  knowledge \nsuch  as  this  so  behaviour  is  not  affected  adversely.  This  was  tested  by  retarding \none  drive  motor of the robot  to give it a  bias to one side. \n\nIn  early  development,  as  the  monitor was  being  tuned,  the  robot  showed  a  tend(cid:173)\nency  to  oscillatory  motion,  thus  exhibiting  undesirable  behaviour  that satisfies  its \ninstincts.  It could, for  example, oscillate back  and forth  at a  corner.  In a  simulated \nenvironment  this  continues  indefinitely.  However,  with  real  mechanics  and  noisy \nanalog sensors  the robot breaks out of this undesirable behaviour. \n\nThese  observations strengthen  the  arguments for  hardware  development  aimed at \nembedded  systems.  The  robot  application is  but  an  example of the  different,  and \noften surprising conditions that pertain in a  \"real\" system.  If neural networks are to \nfind  applications in real-world, low-cost and analog-interface applications, these  are \nthe conditions we  must deal  with,  and appropriate, analog hardware is  the optimal \nmedium for  a solution. \n\n\fCompetence Acquisition  in  an  Autonomous  Mobile  Robot \n\n1037 \n\n5  CONCLUSIONS \n\nThis paper has described  pulse stream neural networks that have been  developed  to \na  system level  to aid development of applications.  We have  therefore  defined  areas \nof strengths of this technology along with suggestions of where  this is  best applied. \nThe strengths of this system include: \n\n1.  Direct  interfacing to analog signals. \n2.  The  ability  to fuse  direct  analog sensor  data with  digital sensor  data pro(cid:173)\n\ncessed  elsewhere  in  the system . \n\n3.  Distributed  processing.  Several  EPC's  may  be  embedded  in  a  system  to \n\nallow  multiple networks  and/or multi layer networks. \n\n4.  The EPC represents  a flexible system level  development environment.  It is \n\neasily reconfigured  for  new  applications or improved chip  technology. \n\n5.  The EPC requires very little computational overhead from the  host system \n\nand can operate  independently  if needed. \n\nA demonstration application of an instinct rule robot has been  presented  highlight(cid:173)\ning the use of neural networks as an interface between  real-world analog signals and \ndigital control. \n\nIn conclusion we  believe that the immediate future of neural analog VLSI is in small \napplications based systems that interface  directly  to  the real-world.  We  see  this as \nthe primary niche area where analog VLSI neural networks will replace conventional \ndigital systems. \n\nAcknow ledgements \n\nThanks are  due  to Ulrich  Nehmzow,  University  of Manchester,  for  discussions  and \ninformation on the instinct-rule controller and the loan of his original robot - Alder. \n\nReferences \n\n[Caudell,  1990]  Caudell,  M.  and  Butler,  C.  (1990).  Naturally  Intelligent  Systems. \n\nMIT  Press,  Cambridge, Ma. \n\n[Jackson,  1994]  Jackson,  G., Hamilton, A.,  and Murray, A.  F.  (1994).  Pulse stream \nVLSI  neural  systems:  into robotics.  In  Proceedings  ISCAS'94,  volume  6,  pages \n375-378. IEEE Press. \n\n[Maren,  1990]  Maren,  A.,  Harston,  C.,  and  Pap,  R.  (1990).  Handbook  of Neural \n\nComputing  Applications.  Academic Press,  San  Diego,  Ca. \n\n[Murray,1992]  Murray,  A.  F.,  Baxter,  D.  J.,  Churcher,  S.,  Hamilton,  A.,  Reekie, \nH.  M., and Tarassenko, L. (1992). The Edinburgh pulse stream implementation of \na  learning-oriented  network  (EPSILON)  chip.  In  Neural Information  Processing \nSystems  (NIPS)  Conference. \n\n[Nehmzow,  1992]  Nehmzow, U.  (1992).  Experiments  in  Competence  Acquisition for \n\nAutonomous  Mobile  Robots.  PhD  thesis,  University  of Edinburgh. \n\n[Widrow,  1988]  Widrow,  B.  (1988).  DARPA  Neural  Network  Study.  AFCEA  In(cid:173)\n\nternational Press. \n\n\f", "award": [], "sourceid": 1134, "authors": [{"given_name": "Geoffrey", "family_name": "Jackson", "institution": null}, {"given_name": "Alan", "family_name": "Murray", "institution": null}]}