{"title": "Comparing the Performance of Connectionist and Statistical Classifiers on an Image Segmentation Problem", "book": "Advances in Neural Information Processing Systems", "page_first": 614, "page_last": 621, "abstract": null, "full_text": "614 \n\nGish and Blanz \n\nComparing  the  Performance  of Connectionist \n\nand  Statistical  Classifiers  on  an  Image \n\nSegmentation  Problem \n\nSheri  L.  Gish  w.  E.  Blanz \nIBM  Almaden  Research  Center \n\n650 Harry Road \n\nSan Jose,  CA  95120 \n\nABSTRACT \n\nIn  this  study,  we  test  the  suitability  of a  connection(cid:173)\n\nIn the development  of an image segmentation system for  real time \nimage processing applications,  we  apply the classical decision anal(cid:173)\nysis  paradigm by  viewing image segmentation as a  pixel classifica.(cid:173)\ntion  task.  We  use  supervised  training  to derive  a  classifier for  our \nsystem  from  a  set  of  examples  of a  particular  pixel  classification \nproblem. \nist  method  against  two  statistical  methods,  Gaussian  maximum \nlikelihood  classifier  and first,  second,  and third degree  polynomial \nclassifiers,  for  the  solution  of a  \"real  world\"  image  segmentation \nproblem  taken  from  combustion  research.  Classifiers  are  derived \nusing  all  three  methods,  and  the  performance  of all  of the  classi(cid:173)\nfiers  on  the  training  data set  as  well  as  on  3  separate  entire  test \nimages  is  measured. \n\nIntroduction \n\n1 \nWe  are  applying  the  trainable  machine  paradigm in  our development  of an  image \nsegmentation  system  to  be  used  in  real  time  image  processing  applications.  We \nview  image  segmentation  as a  classical decision  analysis  task;  each pixel in a  scene \nis  described  by a  set  of measurements,  and  we  use  that  set  of measurements  with \na  classifier  of our choice  to  determine  the  region  or  object  within  a  scene to  which \nthat pixel belongs.  Performing image segmentation as  a  decision  analysis  task pro(cid:173)\nvides  several advantages.  We  can exploit  the inherent trainability found  in decision \n\n\fComparing the Performance of Connectionist and Statistical Classifiers \n\n615 \n\nanalysis  systems  [1 J  and  use  supervised  training to  derive  a  classifier from  a  set  of \nexamples  of a  particular pixel  classification  problem.  Classifiers  derived  using  the \ntrainable machine paradigm will exhibit the property of generalization, and thus can \nbe applied to data representing a set of problems similar to the example problem.  In \nour pixel classification scheme,  the  classifier can be derived  solely  from  the qU8J1ti(cid:173)\ntative characteristics of the problem data.  Our approach eliminates the dependency \non  qualitative  characteristics  of the  problem  data  which  often  is  characteristic  of \nexplicitly  derived classification algorithms  [2,3J. \n\nClassical  decision  8J1alysis  methods  employ  statistical  techniques.  We  have  com(cid:173)\npared  a  connectionist  system to  a  set  of alternative statistical methods  on  classifi(cid:173)\ncation problems in which the classifier is derived using supervised training, 8J1d have \nfound  that  the  connectionist  alternative  is  comparable,  and  in  some  cases  prefer(cid:173)\nable,  to  the statistical alternatives in  terms  of performance on  problems of varying \ncomplexity  [4J.  That  comparison  study  also  8J.lruyzed  the  alternative  methods  in \nterms of cost of implementation of the solution architecture in digital LSI.  hl terms \nof our cost analysis,  the connectionist architectures were much simpler to implement \nthan the statistical architectures for  the more  complex classification problems;  this \nproperty of the connectionist methods makes  them very  attractive implementation \nchoices for  systems requiring hardware implementations for  difficult  applications. \n\nIn  this  study,  we  evaluate  the  perform8J.lce  of a  connectionist  method  and  several \nstatisticrumethods as  the classifier component of our real time image segmentation \nsystem.  The classification problem we  use is  a  \"real world\"  pixel classification task \nusing images  of the  size  (200 pixels by  200  pixels)  and variable data quality typical \nof  the  problems  a  production  system  would  be  used  to  solve.  We  thus  test  the \nsuitability  of the connectionist method for  incorporation in a  system with the per(cid:173)\nformance requirements of our system,  as  well as the feasibility of our exploiting the \nadv8J.ttages  the simple connectionist architectures provide for  systems implemented \nin hardware. \n\n2  Methods \n2.1  The Image Segmentation System \n\nThe  image  segmentation  system  we  use  is  described  in  [5J,  and  summarized  in \nFigure  1.  The  system is  designed  to  perform low  level image  segmentation  in real \ntime;  for  production,  the  feature  extraction  and  classifier  system  components  are \nimplemented in hardware.  The classifier par8J.neters are derived during the Training \nPhase.  A  user  at  a  workstation  outlines  the  regions  or  objects  of  interest  in  a \ntraining  image.  The  system  performs  low  level feature  extraction  on  the  training \nimage,  and  the  results  of the  feature  extraction  plus  the  input  from  the  user  are \ncombined automatically by the system to form a  training data set.  The system then \napplies  a  supervised  training  method  making use  of the  training data set  in  order \nto derive  the coefficients for  the classifier  which  can perform the pixel classification \ntask.  The feature  extraction process is  capable of computing 14 classes  of features \nfor  each pixel;  up to 10 features  with  the  highest  discriminatory power are used to \n\n\f616 \n\nGish and Blanz \n\ndescribe all of the pixels in the image.  TIns selection of features  is based only on an \nanalysis  of the results  of the  feattue  extraction process  and  is  independent  of the \nsupervised  learning paradigm being used  to  derive  the  classifier  [6].  The  identical \nfeature  extraction process  is  applied  in  both  the Training and Running Phases  for \na  particular image segmentation problem. \n\nTraining Images \n\nTest Image \n\nCoefficients \n\nfor \n\nClassifier \n\nTRAINING \n\nPHASE \n\nSegmented \n\nImage \n\nRUNNING \n\nPHASE \n\nFigure 1:  Diagram of the real time  image segmentation system. \n\n2.2  The Image Segmentation  Problem \n\nThe image segmentation problem used in this study is from combustion research and \nis described in [3].  The images are from a  series of images of a  combustion chamber \ntaken  by  a  high  speed  camera during  the  inflammation  process  of a  gas/air  mix(cid:173)\nhue.  The segmentation task is  to determine the area of inflamed gas  in the image; \ntherefore,  the  pixels  in  the  image  are  classified  into  3  different  classes:  cylinder, \nuninflamed  gas,  and  flamed  gas  (See  Figure  2).  Exact  determination  of the  area \nof flamed  gas  is  not  possible  using  pixel  classification  alone,  but  the  greater  the \nsuccess  of the  pixel  classification  step,  the  greater  the  likelihood  that  a  real  time \nimage segmentation system could be used successfully  on  this problem. \n\n2.3  The  Classifiers \n\nThe  set  of  classifiers  used  in  tIns  study  is  composed  of  a  connectionist  classifier \nbased on the Parallel Distributed Processing (PDP) model described in  [7]  and two \nstatistical methods:  a  Gaussian  maximum likelihood  classifier  (a Bayes  classifier), \nand  a  polynomial classifier  based  on  first,  second,  and  third  degree  polynomials. \nTlus  set  of  classifiers  was  used  in  a  general  study  comparing  the  performance  of \n\n\fComparing the Performance of Connectionist and Statistical Classifiers \n\n617 \n\nFigure  2:  The imnge  segmentntion problem  is  to  classify  each imllge  pixel into  1 \nof 3  regions. \n\nthe  alternatives  on  a  set  of classification  problems;  all  of  the  classifiers  as  well  as \nadaptation  procedures  are  described  in  detnil  in  that  study  [4].  Implementation \nand adaptation of nll classifiers in this study was  performed as software simulation. \nThe connectionist  classifier was  implemented in eMU Common Lisp rmming on an \nIBM  RT  workstation. \n\nThe  connectionist  classifier  nrchitecture  is  a  multi-Inyer  feedforwnrd  network  with \none  hidden  layer.  The  network  is  fully  connected,  but  there  nre  only  connections \nbetween  ndjacent  layers.  The  number  of units  in  the  input  and  output  layers  are \ndetermined  by  the  number  of features  in  the  fenture  vector  describing  ench  pixel \nand a  binary encoding scheme for  the class  to which the pixel belongs,  respectively. \nThe  number of units in the hidden  layer  is  an  architectural  \"free  parnmeter.\"  The \nnetwork  used  in  this  study  has  10  units  in  the  input  layer,  12  units  in  the  hidden \nlayer,  and 3 units in  the  outPllt layer. \n\nNetwork activation is  achieved  by  using the  continuous,  nonlinear  logistic  function \ndefined  in  [8].  The  connectionist  adaptation  procedure  is  the  applicntion  of  the \nbackpropagation learning rule also defined in [8].  For this problem, the learning rnte \nTJ  =  0.01  and  the momentum  a  =  0.9;  both  terms  were  held  conshmt  throughout \nadaptntion.  The presentation of all of the patterns ill  the training data set is termed \na  trial;  network weights nnd unit binses were updated after  the presentation of each \npattern during a  trial. \n\nThe  training  data  set  for  this  problem  was  generated  automatically  by  the  image \nsegmentation  system.  This  training  data  set  consists  of approximately  4,000  ten \nelement (feature) vectors (each vector describes  one pixel);  each vector is labeled as \nbelonging to one of the 3 regions of interest in  the imnge.  The training data set  was \nconstructed from one entire  training image,  and is composed of vectors stntistically \nrepresentative of the pixels  in  each of the  3  regions of interest  in  that  image. \n\n\f618 \n\nGish and Dlanz \n\nAll of the  classifiers  tested in  this  study were  adapted from  the same  training data \nset.  The connectionist classifier was defined to be converged for  tlus problem before \nit  was  tested.  Network convergence  is  determined from  the results  of two  separate \ntests.  III  the  first  test,  the  difference  between  the  network  output  and  the  target \noutput  averaged  over  the entire  training data set  has  to reach  a  minimum.  In  the \nsecond  test,  the performance  of the network  in  classifying  the  training data set  is \nmeasured,  and  the  number of misclassifications made  by  the network has  to reach \na  minimum.  Actual network performance in classifying a  pattern is  measured after \npost-processing of the  output  vector.  The real  outputs of each  unit  in  the  output \nlayer  are  assigned  the  values  of 0  or  1  by  application  of a  0.5  decision  threshold. \nIn  our  binary  encoding  scheme,  the  output  vector  should  have  only  one  element \nwith  the  value  1;  that  element  corresponds  to  one  of the  3  classes.  H the network \nproduces an output vector with either more than one element with the value 1 or all \nelements with the value 0,  the pattern generating that output is considered rejected. \nFor the  test  problem in  this  study,  all  of the  classifiers  were  set  to  reject  patterns \nin  the  test  data samples.  All  of the  statistical classifiers  had  a  rejection  threshold \nset  to 0.03. \n\n3  Results \n\nThe performance  of each  of the  classifiers  (connectionist,  Gaussian maximum like(cid:173)\nlihood,  and linear,  quadratic,  and cubic polynomial)  was  measured on the training \ndata set  and  test  data  representing  3  entire  images  taken  from  the  series  of com(cid:173)\nbustion chamber images.  One of those  images,  labeled Inlage  1,  is  the image from \nwhich the training data set was constructed.  The performance of all of the classifiers \nis  summarized in Table  1. \n\nAlthollgh  all  of the classifiers  were  able  to classify  the training data set  with com(cid:173)\nparably few  misclassifications,  the  Gaussian maximum likelihood classifier and the \nquadratic  polynomial classifier  were  unable  to  perform  on  any  of the  3  entire  test \nimages.  The connectionist classifier  was  the only alternative tested in this  study to \ndeliver acceptable performance on all 3  test images;  the connectionist classifier had \nlower error rates  on the test images  than it  delivered on  the  training data sample. \nBoth the  linear polynomial  and  cubic  polynomial classifiers  performed  acceptably \non  the  test  Image  2,  but  then  both  exhibited  high  error  rates  on  the  other  two \ntest images.  For  this  image segmentation problem,  only  the  connectionist  method \ngeneralized from  the training data set  to a  solution with  acceptable performance. \n\nIn Figure 3,  the results from pixel classification performed by the connectionist and \npolynonual classifiers  on all 3  test  images  are  portrayed as  segmented images.  The \nactual  test  images  are included  at  the left  of the figure. \n\n4  Conclusions \n\nOur results demonstrate the feasibility of the application of a  connectionist decision \nanalysis method to the solution of a  ureal world\"  image segmentation problem.  The \n\n\fComparing the Performance of Connectionist and Statistical Classifiers \n\n619 \n\n-~auss;an --] \nClassifier \nError  I Reject \n12.84%  ---0.12%-\n\n94.27% \n\n0.00% \n\n----~~----~~----~ \n\n69.09% \n\n0.01% \n\n88.35% \n\n0.00% \n\n~ata Sel \n\nII \n\nClassifier \n\nConne;;l;on;sl \nError fl  I Rejectb \n1O.40%-~-.64% \n\n,----T \n\n.  . \n\n8.84% \n\nImage  1 C \n\nraInIng \nData \n\n1 \n2 \n3 \n1 \n2 \n3 \nr-~------~~--~-r~~~~~r----\n1 \n2 \n3 \n\nImage  2 \n\n1.53% \n\n1.72% \n\n5.82% \n\nImage  3 \n\n6.31 % \n\n- -::-1.-=-6-=3 %=o- tf---- 1=----\n\n- Polynom;al \nClassifier \n\n--\nDegree  I  Error  I Reject \n1.62% \n1.41% \n1.05% \n4.63% \n3.66% \n0.28% \n2.00% \n0.58% \n0.26% \n5.43 % \n1.41% \n0.28% \n\n'1l.25% \n9.61% \n8.13% \n41.70% \n57.55% \n25 .86% \n12.01% \n68.01 % \n4.68% \n19.68 % \n45.89% \n25.75% \n\n2 \n3 \n\n, ______ _  ~ _ __ _ __  L_  _\n\n_ ____  ~ ______  L_  _____ _  ~ __ ____  ~ ___ ___ ~ ______  __ \n\nflPercent  misclauificatioDi  for  all patterns. \n\nbpercent  of all patterns  rejected. \n\nClmage  from  which  training  data let  was  taken. \n\nTable  1:  A  sununary of the performance  of the c16Ssifiers. \n\ninclusion  of a  connectionist classifier in our supervised segmentation system will al(cid:173)\nlow us to meet our performance requirements under real world problem constraints. \n\nAlthough  the  application  of connectionism  to  the  solution  of  real  time  machine \nvision  problems  represents  a  new  processing  method,  our  solution strategy h6S  re(cid:173)\nmained consistent with the decision analysis paradigm.  Our connectionist cl6Ssifiers \nare derived solely from the quantitative characteristics of the problem data; our con(cid:173)\nnectionist  architecture  thus  remains  simple  and need  not  be re-designed  according \nto  qualitative  characteristics  of each  specific  problem  to  which  it  will  be  applied. \nOur connectionist architecture is independent of the image size;  we have applied the \nidentical architecture  successfully  to images  which range in size from  200  pixels  by \n200  pixels  to  512  pixels by  512  pixels  [9).  In most research to date in which neural \nnetworks  are applied to machine vision,  entire images explicitly are mapped to net(cid:173)\nworks by making each pixel in an image correspond to a  different  unit in a  network \nlayer  (see  [10,11)  for  examples).  This  \"pixel map\"  representation makes  scaling up \nto larger image sizes from the idealized  \"toy\"  research images a  significant problem. \n\nMost  statistical  pattern  classification  methods  require  that  problem  data  satisfy \ntIle  assumptions  of statistical models;  unfortunately,  real  world  problem  data  are \ncomplex and of variable quality and thus rarely can be used to guide the choice of an \nappropriate method for  the solution of a  particular problem a priori.  For the image \nsegmentation problem reported in this study, our cI6Ssifier performance results show \nthat the problem data actually did not satisfy the assumptions behind the statistical \nmodels  underlying  the  Gaussian  maximum  likelihood  classifier  or  the  polynomial \n\n\f620 \n\nGish and Blanz \n\nFigure  3:  The  grey  levels  assigned  to  each  region  nre:  Black  -\nGrey -\nfigure. \n\ncylinder,  Light \nfhnned  gas.  Original images  nre  at  the left  of the \n\nuninflamed  gas,  Grey -\n\nclassifiers.  It  appenrs  that  the  Gaussian  model  least  fits  our  problem  data,  the \npolynomial  classifiers  provide  a  slightly  better  fi t,  and  the  connect.ionist  method \nprovides  the fit  required  for  the  solution  of the  problem.  It is also  notable  that  all \nthe  alternative  m.ethods  in  this  study  could  be  aflapted  to  perform  acceptably  on \nthe  training  data  set,  but  extensive  testing  on  several  different  entire  images  was \nrequired  in  order  to  demonstrate  the  true performance  of the  n1t.ernntive  lllethods \non  the  actual problem.,  rather  than just on  the  trnining data set. \n\nThese results  show  that  a  connectionist method is  a  viable choice for  n  system. such \nas  ours  which  requires  a  simple  nrchitecture readily  implemented  in  hardware,  the \nflexibility  to handle cOl1lpi('x  problems described by large amounts of data,  and  the \nrobustness  to not require problem data to meet, many model assnmptions  11  priori. \n\n\fComparing the Performance of Connectionist and Statistical Classifiers \n\n621 \n\nReferences \n[lJ  R.  O.  Duda a.nd  P.  E.  H6I't.  Pattern  Cla$$ification  and Scene  Analy,i$.  Wiley, \n\nNew  York,  1973. \n\n[2J  W.  E.  Blanz,  J.  L.  C.  Sanz,  and  D.  Petkovic.  Control-free  low-level  image \nsegmentation:  Theory,  architecture,a.nd  experimentation.  In  J.  L.  C.  Sanz, \neditor,  Advance$  of Machine  Vi$ion,  Application$  and Architect-ure\"  Springer(cid:173)\nVerlag,  1988. \n\n[3]  B.  Straub  and  W.  E.  Blanz.  Combined  decision  theoretic  and  syntactic  ap(cid:173)\n\nproach to image  segmentation.  Machine  Vi,ion  and  Application$,  2(1 ):17-30, \n1989. \n\n[4J  Sheri L.  Gish and W. E. Blallz.  Comparing  a  Connectioni$t  Trainable  Clauifier \nwith  Clauical Stati$tical Deci,ion AnalY$i$  Method$.  Research Report RJ 6891 \n(65717),  IBM,  Jtme 1989. \n\n[5]  W.  E.  Bla.nz,  B.  Slmng,  C.  Cox,  W.  Greiner,  B.  Dom,  a.nd  D.  Petkovic.  De$ign \nand  implementation  of  a  low  level  image  ,egmentation  architecture  - LISA. \nResearch Report  RJ 7194  (67673),  IBM,  December  1989. \n\n[6]  W.  E.  BI611z.  Non-p6I'ametric feature  selection for  multiple  class  processes.  In \n\nProc.  9th  Int.  Con/.  Pattern Recognition,  Rome, Italy,  Nov.  14-17 1988. \n\n[7J  David E.  Rumelhart, J61ues  L.  McClelland,  et a1.  Parallel Di$tributed Proceu(cid:173)\n\ning.  MIT  Press,  C61ubridge,  Massachusetts,  1986. \n\n[8]  David E.  Rumelhart, Geoffrey E. Hinton, and Ronald J. Willia.ms.  Le6I'nillg in(cid:173)\nternal representations by error propagation.  In David E.  Rumelllart,  James L. \nMcClell611d,  et  aI.,  editors,  Parallel  Di,tributed  Proce66ing,  chapter  8,  MIT \nPress,  Cambridge,  Massachusetts,  1986. \n\n[9]  W.  E.  Bla.nz 6l1d Sheri L.  Gish.  A  Connectioni,t Clauifier Architecture Applied \nTo  Image  Segmentation.  Rese6I'ch  Report  RJ  7193  (67672),  IBM,  December \n1989. \n\n[10}  K.  Fukushima,  S.  Miyake,  and T.  Ito.  Neocognitron:  a  neura.lnetwork model \nfor a mechanism of visual pattern recognition.  IEEE  Tran$actio1t$  on Sy,tem\" \nMan,  and  Cybernetic$,  SMC-13(5):826-834,  1983. \n\n[U}  Y.  Hirai.  A  model  of humau  associative  processor. \n\nIEEE  Tran$action$  on \n\nSy,tem$,  Man,  and  Cybernetic$,  SMC-13(5):851-857,  1983. \n\n\f", "award": [], "sourceid": 236, "authors": [{"given_name": "Sheri", "family_name": "Gish", "institution": null}, {"given_name": "W.", "family_name": "Blanz", "institution": null}]}