{"title": "Optimal Movement Primitives", "book": "Advances in Neural Information Processing Systems", "page_first": 1023, "page_last": 1030, "abstract": null, "full_text": "Optimal Movement  Primitives \n\nTerence D.  Sanger \n\nJet Propulsion Laboratory \n\n(818)  354-9127 tds@ai.mit.edu \n\nMS  303-310 \n\n4800  Oak  Grove  Drive \nPasadena, CA  91109 \n\nAbstract \n\nThe  theory  of Optimal  Unsupervised  Motor  Learning  shows  how \na  network  can  discover  a  reduced-order  controller for  an  unknown \nnonlinear system by  representing only  the most significant  modes. \nHere,  I extend the theory  to apply to command sequences,  so that \nthe most significant  components discovered  by  the  network  corre(cid:173)\nspond  to  motion  \"primitives\".  Combinations of these  primitives \ncan  be  used  to  produce  a  wide  variety  of  different  movements. \nI  demonstrate  applications  to  human  handwriting  decomposition \nand  synthesis,  as  well  as  to  the  analysis  of  electrophysiological \nexperiments  on  movements resulting  from  stimulation of the  frog \nspinal cord. \n\n1 \n\nINTRODUCTION \n\nThere  is  much  debate  within  the  neuroscience  community  concerning  the  inter(cid:173)\nnal  representation  of movement, and  current  neurophysiological  investigations  are \naimed  at  uncovering  these  representations. \nIn  this  paper,  I  propose  a  different \napproach  that  attempts  to  define  the  optimal internal  representation  in  terms  of \n\"movement primitives\" , and I compare this representation with the observed behav(cid:173)\nior.  In  this  way,  we  can  make strong  predictions  about  internal signal  processing. \nDeviations  from  the  predictions  can  indicate  biological  constraints  or  alternative \ngoals that cause  the biological system to be suboptimal. \n\nThe concept of a motion primitive is not as well defined as that of a sensory primitive \n\n\f1024 \n\nTerence Sanger \n\nu \n\ny \n\np \n\nz \n\nFigure 1:  Unsupervised  Motor Learning:  The plant P  takes inputs u  and produces \noutputs y.  The sensory map C produces intermediate variables z, which are mapped \nonto the  correct  command inputs by  the motor network  N. \n\nwithin the visual system, for example.  There is no direct equivalent to the  \"receptive \nfield\"  concept that has allowed interpretation of sensory recordings.  In  this paper, I \nwill propose  an internal model that involves both motor receptive fields  and a set of \nmovement primitives which  are combined using a  weighted sum to produce  a  large \nclass  of movements.  In  this  way,  a  small  number  of well-designed  primitives  can \ngenerate  the full  range of desired  behaviors. \n\nI  have  previously  developed  the concept  of \"optimal unsupervised  motor learning\" \nto investigate optimal internal representations  for  instantaneous motor commands. \nThe  optimal  representations  adaptively  discover  a  reduced-order  linearizing  con(cid:173)\ntroller for  an  unknown nonlinear  plant.  The  theorems  give  the optimal solution in \ngeneral,  and  can  be  applied  to  special  cases  for  which  both  linear  and  nonlinear \nadaptive  algorithms exist  (Sanger  1994b).  In  order  to  apply  the  theory  to  com(cid:173)\nplete  movements it needs  to be  extended slightly, since in general movements exist \nwithin  an  infinite-dimensional  task  space  rather  than  a  finite-dimensional  control \nspace.  The  goal  is  to  derive  a  small  number  of primitives  that  optimally encode \nthe  full  set  of observed  movements.  Generation  of the  internal  movement  primi(cid:173)\ntives  then  becomes  a  data-compression  problem,  and  I  will  choose  primitives that \nminimize the resultant  mean-squared error. \n\n2  OPTIMAL  UNSUPERVISED  MOTOR LEARNING \n\nOptimal Unsupervised  Motor  Learning is  based on  three  principles: \n\n1.  Dimensionality Reduction \n2.  Accurate  Reduced-order  Control \n3.  Minimum Sensory error \n\nConsider the system shown in figure  1.  At time t, the plant  P  takes motor inputs u \nand produces sensory  outputs y.  A sensory  mapping C  transforms the raw sensory \ndata y  to an intermediate representation  z.  A  motor mapping takes  desired  values \nof z  and computes the  appropriate command u such  that CPu =  z.  Note  that the \n\n\fOptimal  Movement  Primitives \n\n1025 \n\nloop in the figure  is not a feedback-control  loop, but is intended to indicate the flow \nof information.  With this  diagram in mind, we  can write  the three  principles  as: \n\n1.  dim[z]  < dim[y] \n2.  GPNz=z \n3.  IIPNGy - yll  is  minimized \n\nWe  can  prove the following theorems  (Sanger  1994b): \nTheorem  1:  For  all  G  there  exists  an  N  such  that  G P N z  =  z.  If G  is  linear  and \np- 1  is  linear,  then  N  is linear. \n\nTheorem  2:  For any G, define an invertible map C such that GC-l =_1 on range[G]. \nThen  liP NGy - yll  is  minimized when  G  is  chosen  such  that  Ily - G-1GII  is  mini(cid:173)\nmized.  If G  and P  are linear and the singular value decomposition of P  is given  by \nLT SR,  then  the optimal maps are  G = Land N  = RT S-I. \nFor the discussion of movement, the linear case  will be the most important since in \nthe nonlinear case  we  can use unsupervised motor learning to perform dimensional(cid:173)\nity reduction  and linearization of the plant at each time t.  The movement problem \nthen becomes  an infinite-dimensional linear problem. \n\nPreviously,  I  have  developed  two  iterative  algorithms  for  computing  the  singular \nvalue decomposition from input/output samples (Sanger 1994a).  The algorithms are \ncalled the  \"Double Generalized  Hebbian Algorithm\"  (DGHA)  and the  \"Orthogonal \nAsymmetric Encoder\"  (OAE).  DGHA is  given by \n\n8G \n8NT \n\n'Y(zyT  - LT[zzT]G) \n'Y(zuT - LT[zzT]NT ) \n\nwhile  OAE is  described  by: \n\n'Y(iyT  - LT[ZiT]G) \n'Y(Gy - LT[GGT]z)uT \n\nwhere  LT[  ]  is  an  operator  that  sets  the  above  diagonal  elements  of its  matrix \nargument to  zero,  y  =  Pu,  z  =  Gy,  z  =  NT u,  and  'Y  is  a  learning  rate  constant. \nBoth algorithms cause G to converge to the matrix of left singular vectors of P, and \nN  to converge to the matrix of right singular vectors  of P  (multiplied by a diagonal \nmatrix for  DGHA) . DGHA is  used  in the examples below. \n\n3  MOVEMENT \n\nIn  order  to extend  the  above  discussion  to  allow  adaptive  discovery  of movement \nprimitives, we  now consider  the plant P  to be a mapping from command sequences \nu(t)  to sensory  sequences  y(t).  We  will  assume  that  the  plant  has  been  feedback \nlinearized  (perhaps  by  unsupervised  motor  learning).  We  also  assume  that  the \nsensory  network  G  is  constrained  to  be  linear.  In  this  case,  the  optimal motor \nnetwork  N  will  also be linear.  The intermediate variables  z  will  be represented  by \na  vector.  The sensory  mapping consists  of a  set  of sensory  \"receptive fields\"  gi(t) \n\n\f1026 \n\nTerence Sanger \n\nA .. \n\nMotor Map \n\nSensory Map \n\nI  n1(t)zl  +- Zl  +-\n\n---lEt): \n\n: \n\n\\  nn(~)~ ~ ~n +-\n\nFigure  2:  Extension  of  unsupervised  motor  learning  to  the  case  of  trajectories. \nPlant input and output  are  time-sequences  u(t)  and  y(t).  The sensory  and motor \nmaps now consist  of sensory  primitives gi(t)  and motor primitives ni(t). \n\nsuch  that \n\nZi  = J gj(t)y(t)dt =< gily > \n\nand the motor mapping consists of a set of \"motor primitives\"  ni(t) such that \n\nu(t) = L ni(t)zi \n\ni \n\nas in figure  2.  If the plant is equal to the identity (complete feedback  linearization), \nthen gi(t)  =  ni(t).  In  this  case,  the optimal sensory-motor primitives are  given  by \nthe eigenfunctions of the  autocorrelation function of y(t) .  If the  autocorrelation is \nstationary, then the infinite-window eigenfunctions will be sinusoids.  Note  that the \noptimal primitives depend both on the plant P  as  well as the statistical distribution \nof outputs  y(t). \nIn  practice, both u(t)  and y(t)  are sampled at discrete  time-points {tic}  over a finite \ntime-window, so that the plant input and output is in actuality a long vector.  Since \nthe plant is linear, the optimal solution is given by the singular value decomposition, \nand either the DGHA or OAE algorithms can be used directly.  The resulting sensory \nprimitives  map the  sensory  information  y(t)  onto  the  finite-dimensional  z,  which \nis  usually  a  significant  data  compression.  The  motor  primitives  map  Z  onto  the \nsequence  u(t),  and  the  resulting  y(t)  =  P[u(t)]  will  be  a  linear  projection  of y(t) \nonto the space spanned by the set {Pni(t)}. \n\n4  EXAMPLE  1:  HANDWRITING \n\nAs  a simple illustration, I examine the case of human handwriting.  We can consider \nthe  plant  to  be  the  identity  mapping from  pen  position  to  pen  position,  and  the \n\n\fOptimal  Movement  Primitives \n\n1027 \n\n1. \n\n2. \n\n3. \n\n4. \n\n5. \n\n6. \n\n7. \n\n8. \n\nFigure 3:  Movement primitives for sampled human handwriting. \n\n\f1028 \n\nTerence Sanger \n\nhuman  to  be  taking  desired  sensory  values  of pen  position  and  converting  them \ninto  motor  commands  to  move  the  pen.  The  sensory  statistics  then  reflect  the \nset of trajectories used  in producing handwritten letters.  An optimal reduced-order \ncontrol system can be designed based on the observed statistics, and its performance \ncan be  compared to human performance. \n\nFor  this  example,  I  chose  sampled  data from  87  different  examples  of lower-case \nletters  written  by  a  single  person,  and  represented  as  horizontal  and  vertical  pen \nposition at each point in time.  Blocks of 128 sequential points were used for training, \nand 8 internal variables Zi  were used for each of the two components of pen position. \nThe training set  consisted of 5000 randomly chosen  samples.  Since  the plant is  the \nidentity,  the  sensory  and  motor primitives  are  the  same,  and  these  are  shown  as \n\"strokes\"  in figure  3.  Linear combinations of these strokes  can be used  to generate \npen paths for  drawing lowercase  letters.  This is  shown in figure  4,  where  the word \n\"hello\"  (not  present  in  the  training set)  is  written  and  projected  using  increasing \nnumbers of intermediate variables Zi.  The bottom of figure  4 shows the sequence of \nvalues of Zi  that was  used  (horizontal component only). \n\nGood  reproduction  of  the  test  word  was  achieved  with  5  movement  primitives. \nA  total  of 7  128-point segments  was  projected ,  and  these  were  recombined  using \nsmooth 50%  overlap .  Each  segment  was  encoded  by  5  coefficients  for  each  of the \nhorizontal  and  vertical  components,  giving  a  total  of 70  coefficients  to  represent \n1792 data points  (896  horizontal and  vertical components) , for  a  compression ratio \nof 25 :1. \n\n5  EXAMPLE  2:  FROG  SPINAL  CORD \n\nThe  second  example models  some interesting  and  unexplained  neurophysiological \nresults  from  microstimulation of the frog spinal cord.  (Bizzi  et  al.  1991)  measured \nthe  pattern  of forces  produced  by  the  frog  hindlimb  at  various  positions  in  the \nworkspace during stimulation of spinal interneurons.  The resulting force-fields often \nhave  a stable\" equilibrium point\" , and in some cases  this equilibrium point follows \na  smooth  closed  trajectory  during  tonic stimulation of the  interneuron.  However, \nonly  a  small number of different  force  field  shapes  have  been  found,  and  an  even \nsmaller  number  of different  trajectory  types.  A  hypothesis  to  explain  this  result \nis  that larger  classes  of different  trajectories  can be formed  by  combining the  pat(cid:173)\nterns  produced  by  these  cells.  This  hypothesis  can  be  modelled using  the  optimal \nmovement primitives described  above. \n\nFigure 5a shows a simulation of the frog leg.  To train the network,  random smooth \nplanar movements were made for 5000 time points.  The plant output was considered \nto be 32  successive  cartesian endpoint  positions, and the plant input was  the time(cid:173)\nvarying force  vector  field.  Two  hidden  units  Z  were  used.  In  figure  5b  we  see  an \nexample of the two equilibrium point trajectories  (movement primitives) that were \nlearned by DG HA. Linear combinations of these  trajectories account for over 96% of \nthe variance of the training data, and they can approximate a large class of smooth \nmovements .  Note that many other pairs of orthogonal trajectories  can accomplish \nthis,  and different  trials often  produced  different orthogonal trajectory shapes. \n\n\fOptimal  Movement  Primitives \n\n1029 \n\nOriginal \n\n1. \n\n2 \n\n3. \n\nCoefficients \n\n5.~ \n\n8.~~ \n\nFigure 4:  Projection of test-word  \"hello\"  using increasing numbers of intermediate \nvariables Zi. \n\n\f1030 \n\no. \n\nWorkSpace \n\nb. \n\nTerence Sanger \n\nFigure 5:  a.  Simulation of frog  leg configuration.  b.  An example of learned optimal \nmovement primitives. \n\n6  CONCLUSION \n\nThe examples  are  not meant to  provide  detailed  models of internal  processing  for \nhuman or  frog  motor control.  Rather,  they  are  intended  to  illustrate the  concept \nof optimal primitives and  perhaps guide the search for  neurophysiological and  psy(cid:173)\nchophysical correlates  of these  primitives.  The first  example shows  that generation \nof the lower-case  alphabet  can  be  accomplished with  approximately 10  coefficients \nper  letter,  and that this covers  a  considerable range of variability in  character pro(cid:173)\nduction.  The second  example demonstrates  that  an  adaptive  algorithm allows  the \npossibility for  the frog  spinal cord to  control movement using  a  very small number \nof internal variables. \n\nOptimal  unsupervised  motor  learning  thus  provides  a  descriptive  model  for  the \ngeneration of a large class of movements using a compressed internal description.  A \nset of fixed  movement primitives can be combined linearly to produce the necessary \nmotor commands, and  the optimal choice of these  primitives assures  that the error \nin the resulting movement will  be minimized. \n\nReferences \n\nBizzi  E.,  Mussa-Ivaldi F.  A.,  Giszter S.,  1991,  Computations underlying the execu(cid:173)\ntion of movement:  A biological perspective,  Science,  253:287-29l. \nSanger  T.  D.,  1994a,  Two  algorithms  for  iterative  computation  of  the  singular \nvalue decomposition from input/output samples,  In Touretzky  D., ed.,  Advances in \nNeural Information  Processing  6,  Morgan  Kaufmann, San Mateo,  CA,  in  press. \nSanger  T.  D.,  1994b,  Optimal unsupervised  motor learning,  IEEE  Trans.  Neural \nNetworks,  in  press. \n\n\f", "award": [], "sourceid": 927, "authors": [{"given_name": "Terence", "family_name": "Sanger", "institution": null}]}