{"title": "Exploiting Syllable Structure in a Connectionist Phonology Model", "book": "Advances in Neural Information Processing Systems", "page_first": 612, "page_last": 618, "abstract": null, "full_text": "Exploiting Syllable Structure \n\nin a Connectionist Phonology Model \n\nDavid S. Touretzky  Deirdre W. Wheeler \n\nSchool of Computer Science \nCarnegie Mellon University \nPittsburgh, PA  15213-3890 \n\nAbstract \n\nIn  a  previous  paper  (Touretzky  &  Wheeler,  1990a) we showed  how  adding  a \nclustering operation to a connectionist phonology model produced a parallel pro(cid:173)\ncessing account of certain \"iterative\" phenomena.  In this paper we show how the \naddition of a second structuring primitive, syllabification, greatly increases  the \npower of the model.  We present examples from  a non-Indo-European language \nthat appear to require rule ordering to at least a depth of four.  By adding syllab(cid:173)\nification circuitry to structure the model's perception of the input string, we are \nable to handle these examples with only two derivational steps.  We conclude that \nin phonology, derivation can be largely replaced by structuring. \n\n1 \n\nIntroduction \n\nIn linguistics a grammar  is  an  abstract formal  system  describing a  language.  The  term \npsycho-grammar  has  been  suggested  for  systems  that  express  the linguistic knowledge \nthat actually exists in speakers' heads (George,  1989).  Psycho-grammars may differ from \ngrammars  as  a result of performance demands, limited memory capacity, or other aspects \nof mental  representations.  Psycho-grammars  are  still  somewhat  abstract,  in  that  they \nare  concerned  with  mental rather than  physical phenomena.  The  term physio-grammar \n(George, 1989) refers to the the physical representation of grammatical knowledge in neural \nstructures, such as  (perhaps) synapse strengths.  Detailed proposals for physio-grammars \ndo  not yet exist;  the field  of neurolinguistics is  insufficiently  advanced  to  support such \nproposals at present. \n\nWe  are  developing a  theory  of phonology that is  compatible  with  gross  constraints  on \nneural processing and cognitive plausibility.  Our research, then, is on the construction of \npsycho-grammars at the phonological level.  We use a connectionist model to demonstrate \n\n612 \n\n\fExploiting Syllable Structure in a Connectionist Phonology Model \n\n613 \n\nM-level \n\nI \n\nM-P Rules \n\nP-level \n\nI \n\nP-F Rules \n\n- - - - - - .  ...... 1  F-level \n\nFigure 1:  Structure of the model. \n\nthe computational feasibility of the psycho-grammar architecture we propose. In this paper \nwe show how the addition of syllabification as a primitive operation greatly increases the \nscope and power of the model at little computational cost. \n\n2  Structure of the Model \n\nOur model. shown in Figure 1, has three levels of representation. Following Lakoff (1989), \nthey are labeled M, P, and F. The M, or morpho-phonemic level, is a sequence of phonemes \nconstructed by concatenating abstract underlying representations of morphemes.  The P, or \nphonemic level,  is  an  intermediate representation that is  constrained to hold syllabically \nwell-formed strings. The F, or phonetic level, is the surface level representation:  a sequence \nof phonetic segments.  Derivations are performed by mapping strings from  M  to P level, \nand then  from  P  to F  level,  under the control of a  set of language-specific rules.  These \nrules alter the mapping in various ways to effect processes such as voicing assimilation and \nvowel harmony. \n\nThe model  has  a  number of important constraints.  Rules at a  given level  (M-P or P-F) \napply  in  a  single  parallel  step  during  the  mapping  from  one  level  to  the  next.  There \nis  no  iterative rule application.  \"Iterative\"  processes  are  instead  handled  by a  parallel \nclustering mechanism described in Touretzky &  Wheeler (1990a,1991). The connectionist \nimplementation uses limited-depth, strictly feed-forward Circuitry, so the model has minimal \ncomputational complexity. \n\nAnother very important constraint is that only two levels of derivation are provided, M-P \nand P-F, so there is  no room for  the long chains of ordered rules  that other phonological \ntheories permit.  However, in standard analyses some languages appear to require long rule \nchains.  The problem for those who want to eliminate such chains on grounds of cognitive \nimplausibility 1 is to reformulate existing linguistic analyses to account for the data in some \n\n1 Here we are referring to Goldsmith (1990) and Lakoff (1989), as well as our own work. \n\n\f614 \n\nTouretzky and Wheeler \n\nI hro+aht fi/ \nhro  ht u \nhr6  htu \nr6  ht u \nr6  hdu \n[r6  hdu] \n\n\"he has disappeared\" \nvowel deletion \nstress \ninitial h-deletion \npre-son.  voicing \n\nI ':\\+k+hrek+?1 \n~khrek? \n, \nA  k  hreke? \n, \nA  k  hrege? \n[~  k  hrege?] \n\n\"I will push it\" \nstress \nepenthesis \npre-son.  voicing \n\nFigure 2:  Two Mohawk derivations. \n\nother way.  This is not always easy to do, especially in our model, which is  more tightly \nconstrained than  either the Goldsmith or Lakoff proposals.  Such reformulations help us \nto see how  psycho-grammar diverges  from  grammar when computational constraints are \ntaken into consideration. \n\n3  A Problem From Mohawk \n\nIn Mohawk, an American Indian language, stress is placed on the penultimate syllable of a \nword.  Since there are processes in Mohawk that add and delete vowels from words, their \ninteraction with the stress  rule is  problematic.  Figure 2 shows  two Mohawk derivations \nin a standard generative account.2  The first  example shows  us  that vowel deletion must \nprecede  stress  assignment.  The penultimate vowel Ia!  in the underlying form  does  not \nappear in the surface form  of the word.  Instead stress is assigned to the preceding vowel, \n10/, which is is the penultimate vowel in the surface form.  The second example shows that \nstress assignment must precede vowel epenthesis (insertion), because the epenthetic leI that \nappears in the surface form is not counted when determining the penultimate vowel.  Since \nthe epenthetic Ie! is also the trigger for presonorant voicing in this example, we see that \nvoicing must be ordered after vowel epenthesis.  Together these two examples indicate the \nfollowing rule ordering:  Vowel  deletion < Stress  < Epenthesis  < Pre-sonorant voiCing. \nBut this is a depth of four, and our model permits only two levels of derivation. We therefore \nmust produce an alternative account of these four processes that requires fewer derivations. \nTo  do  so,  we rely  on  three  features  of the model:  parallel  rule application,  multi-level \nrepresentations, and a structuring primitive:  syllabification. \n\n4  Representation of Syllable Structure \n\nMost insertion and deletion operations are syllabically-motivated (Ito,  1989).  By adding \na  syllabification  mechanism  to  our  model,  we  can  replace  certain  derivational  (string(cid:173)\nrewriting)  steps  with  more  constrained  and  perhaps  cognitively less  taxing  structuring \nsteps.  Linguists represent syllables as tree structures, as in the left portion of Figure 3. The \nnucleus of the syllable is normally a vowel.  Any preceding consonants form the onset, and \nany following consonants the coda.  The combined nucleus and coda make up the rime.  In \nthe middle portion of Figure 3 the syllabic structure of the English word ''tokens'' (phonetic \ntranscription [tok\u20acnz]) \nis  shown in this hierarchically structured form.  The right portion \nshows  how we encode the same information in our model  USing  a set of onset, nucleus, \n\n2These examples, derived from  Halle &  Clements (1983), are cited in Lakoff (1989).  We thank \n\nMarianne Mithun (p.c.)  for correcting an error in the original data. \n\n\fExploiting Syllable Structure in a Connectionist Phonology Model \n\n615 \n\nsyllable //\\ \n\nnucleus  coda \n\nonset \n\na \n\na \n\n11  /)\" \n\nN \n\nN \n\nC \n\n0 \n\n0 \n\nI  /'\\ \nz \n\nn \n\n\u00a3 \n\n0 \n\nk \n\n+ \n\nk \n+ \n\n0 \n\n+ \n\n\u00a3 \n\nn \n\nz \n\n+ \n\n+ \n\n+ \n\nonset: \n\nnucleus: \n\ncoda: \n\nFigure 3:  Representations for syllable structure. \n\nM: \nonset: \nnucleus: \ncoda: \n\nP: \n\nF: \n\nhroahtii \n++ \n+ \n+  + \n\n+ \n\nhr 6  ht ii \n\nr 6  hdii \n\nvowel del. \n\nstress (M -P) \n\nh-del.; pre-son. \nvoicing (P-F) \n\ni\\khrek  ? \n\n++  + \n+ \n\nM: \nonset: \nnucleus:  + \n+ \ncoda: \n, \ni\\khr eke? \n\nP: \n\n+ \n\nepenthesis \n\nstress (M-P) \n\npre-son. \n\nF: \n\n~khrege? \n\nvoicing (P-F) \n\nFigure 4:  Our solution to the Mohawk problem. \n\nand coda bits, or ONe bits for short.  We have no explicit representation for rimes, but this \ncould be added if necessary. \n\nIn Mohawk, the vowel deletion and epenthesis processes are both syllabically motivated. \nVowel deletion enforces a constraint against branching nuclei. 3  Epenthesis inserts a vowel \nto break up a word-final consonant cluster (jk/ followed by glottal stop f!f) that would be an \nillegal syllable coda.  Our contention is that syllabification operates on the M-Ievel string \nby setting the associated ONe bits in such a way that the P-Ievel string will be syllabically \nwell-formed.  The ONe bits share control with the M-P rules of the mapping from M to P \nlevel. \n\nEvery M -level segment must have one of its ONe bits set in order to be mapped to P-Ievel. \nThus, the syllabifier can cause a vowel to be deleted at P simply by failing to set its nucleus \nbit, as occurs for the Ia! in /hroahtiil in Figure 4. For the Ii\\khrek? I example, note in Figure 4 \nthat the /kI has  been marked as  an  onset by the syllabifier and the f!1  as  a coda;  there is \nno intervening nucleus.  This automatically triggers an insertion by the M-P map, so that a \nvowel will appear between these two segments at P-Ievel.  The vowel chosen is the default \nor \"unmarked\" vowel for that particular language; for Mohawk it is leI.  For further details \nof the syllabification algorithm, see Touretzky &  Wheeler (1990b). \n\nThe left half of Figure 5 shows our formulation of the Mohawk stress rule, which assigns \nstress  to the penultimate nucleus of a  word.  Rather than looking directly at the M-Ievel \nbuffer, the rule looks at the ''projection'' of the nucleus tier.  By this we mean the M-Ievel \nsubstring consisting of those segments  whose nucleus  bit is  set.  The #  symbol indicates \na  word boundary.  Since vowels  deleted  by the syllabifier have no  nucleus  bit set,  and \n\n3This constraint is not shared by all languages. Furthermore, deletion is only one possible solution; \nanother would be to  insert a consonant or glide,  such as  Iwl,  to  separate the  vowels into  different \nsyllables.  Each language makes its own choices about how constraint violations are to be repaired. \n\n\f616 \n\nTouretzky and Wheeler \n\nM[nucleus]: \n\nP: \n\n[] \nI \n\n[+stress] \n\n[] \n\n# \n\nP: \n\nF: \n\n[-son] \n\n[+son] \n\nI \n\n[+voice] \n\nFigure 5:  Rules for Mohawk stress (M-P) and presonorant voicing (P-F). \n\nepenthetic vowels  that will be inserted by the syllabifier have no nucleus  bit at M-Ievel, \ninsertion and deletion processes can proceed in parallel with stress assignment.  At P-Ievel, \nall that's left to be done in this example is  pre-sonorant voicing, handled by the P-F rule \nshown in the right half of the figure. \n\n5  More Complex Stress Rules \n\nIn Mohawk,  stress falls  on the penultimate syllable regardless of the internal structure of \nthe syllable.  This stress assignment rule is quite simple compared to some other languages. \nFor example,  \"quantity sensitive\" languages  make distinctions among  syllable types  for \npurposes of stress assignment.  A syllable consisting of an optional onset and a single, short \nvowel in the rime is normally said to be \"light,\" while syllables  with codas  and/or long \nvowels (often represented as  double nuclei) are designated \"heavy,\" and typically attract \nstress.  Thus, for example, in Aguacatec Mayan (Hayes, 1981) stress falls on the rightmost \nsyllable with a long vowel, otherwise the final syllable. \n\nIn  order to  account  for  syllable weight  distinctions we introduce an  additional  level  of \nrepresentation, as illustrated in Figure 6 using C and V to represent consonants and vowels, \nrespectively. The \"mora\" bit is activated for all segments that contribute to syllable weight \nin the language.  In this particular language only vowels are important for determining the \nweight of syllables, so the mora bit is activated for all and only the vocalic segments.  Once \nmoras  have been  identified,  universal  principles  come  into play,  and bits  for  \"syllable\" \nand  \"heavy syllable\" are  set.  The syllable bit is  activated  for the first of a  sequence of \none or more moras; the heavy syllable bit is activated for syllables containing two or more \nmoras.  With this enriched representation, the stress patterns of quantity-sensitive languages \ncan  be straightforwardly generated.  To stress the last heavy syllable, we assign  [+stress] \nto  segments  on the heavy  syllable tier that have word boundaries  to  their right.  (Word \nboundaries must be projected down to the heavy syllable tier for this purpose.) \n\nLanguages like Yana (Hayes,  1981), in which both long vowels and codas make syllables \nheavy, have a slightly different representation at the mora level.  In these languages, coda \nconsonants as well as vocalic segments trigger the activation of the mora bit, as illustrated \nin Figure 7.  Here again, while specification of what counts as a mora is a language-specific \nparameter, once the mora bits are set the syllable and heavy syllable representations follow \nfrom  universal prinCiples.  The mora bit is activated for any segment which has either the \nnucleus or coda bit set, essentially collapsing the nucleus and coda tiers.  The Yana stress \nrule targets  the leftmost heavy syllable in a word, no matter how far  it might occur from \nthe initial word boundary, or the first syllable if none are heavy.  The latter case requires a \nseparate rule with a slightly more complex environment; rules of this form are discussed in \nWheeler & Touretzky (1991). \n\n\fExploiting Syllable Structure in a Connectionist Phonology Model \n\n617 \n\n#  C  v  c  v  c  v  V  c  V  c  V  V  C \n\n# \n\nonset \nnucleus \ncoda \n\n+ \n\nmora \nsyllable \nheavy syllable \n\n+ \n\n+ \n\n+ \n\n+ \n+ \n\n+ \n\n+ \n+ \n\n+ \n\n+ \n\n+ \n\n+ \n+ \n\n+  + \n\n+  + \n+ \n+ \n\n+ \n\n+  + \n\n+  + \n+ \n+ \n\nFigure 6:  Long vowels make syllables heavy in Aguacatec Mayan. \n\n#  C  V  C  V  C  C  V  C  V  V  C  V  V  C \n\n# \n\nonset \nnucleus \ncoda \n\n+ \n\nmora \nsyllable \nheavy syllable \n\n+ \n\n+ \n\n+ \n+ \n\n+ \n\n+ \n\n+  + \n+ \n+ \n\n+ \n\n+ \n\n+ \n\n+ \n+ \n\n+  + \n\n+  + \n+ \n+ \n\n+ \n\n+  + \n\n+ \n\n+  +  + \n+ \n+ \n\nFigure 7:  Long vowels or codas make syllables heavy in Yana. \n\n6  Discussion \n\nFor the linguist, it is  interesting to see how structuring operations such as  clustering and \nsyllabification can take some of the pressure off derivation, thereby allowing strict limits to \nbe maintained on derivational depth.  But what is the significance of this work for connec(cid:173)\ntionists?  Unlike most other attempts to model phonological processes in neural networks, \nwe demonstrate the influence computational modeling can  have on the development of a \nlinguistic theory.  In designing a system for expressing linguistic processes, there must be \nsome sort of cost metric to determine which operations are computationally feasible and \nwhich are not.  A connectionist implementation provides a natural cost metric:  size (depth, \nfanout, component count) of the required threshold logic circuity. \n\nIt is doubtful that the structure of our model corresponds to that of some cortical language \narea, and we reject any simplistic analogy between threshold logic units and neurons. Using \ncircuit complexity as  a cost metric can be independently justified on grounds of simplicity \nand theoretical elegance.  If one measures cost in some more abstract way, there is a danger \nthat  computationally expensive  mechanisms  may  lurk  beneath  the  grammar's  apparent \nsimpliCity.  An example is  the local rule ordering proposal of Anderson (1974), in which \nexplicit rule ordering is eliminated by introducing a much  more complex mechanism  for \ndetermining, on a case-by-case basis, the order in which rules should apply. \n\nIf the  mental  representation  of utterances  is  fundamentally  different  from  the  discrete \nsymbolic form  we've assumed.4  we may be using the wrong cost metric for determining \ncognitive plausibility.  However. we are constrained. like everyone else. to work within the \ncomputational frameworks  that are presently available. \n\n4Por example:  if phonetic strings tum out to be represented in the brain as chaotic trajectories in \n\na high dimensional dynamical system, or something equally exotic. \n\n\f618 \n\nTouretzky and Wheeler \n\nThere remains the question of why structuring should be preferred over derivation.  First, \nsince some mutation processes are sensitive to syllabic structure, this  information would \nhave to be computed even  if insertions and deletions weren't handled by the syllabifier. \nSecond, structuring is a highly constrained operation; it merely annotates an existing string \nto reflect constituency relationships, whereas derivations can make arbitrary changes to a \nstring.  We therefore assume that derivations have a higher cognitive cost, despite the fact \nthat they can be computed fairly efficiently in our model by the mapping matrix described \nin Touretzky  &  Wheeler  (1991).  Finally,  adding extra derivational  levels  increases  the \ndifficulty of phonological rule induction, a topic of current research. \n\nAcknowledgements \n\nThis work was  sponsored by a grant from  the Hughes  Aircraft Corporation, by National \nScience Foundation grant EET-8716324, and by the Office of Naval Research under contract \nnumber NOOOI4-86-K-0678. \n\nReferences \n\nAnderson, S. R. (1974) The Organization of Phonology.  New York:  Academic Press. \n\nGeorge,  A.  (1989) How  not to become confused  about linguistics.  In  A.  George  (ed.), \nReflections on Chomsky, 90-110. Oxford, UK: Basil Blackwell. \n\nGoldsmith,  J.  A.  (1990)  Autosegmental  and  Metrical  Phonology.  Oxford,  UK:  Basil \nBlackwell. \nHalle,  M.,  and  Clements,  G.  N.  (1983) Problem  Book in  Phonology:  A  Workbook for \nIntroductory Courses in Linguistics and Modern Phonology.  Cambridge, MA:  The MIT \nPress. \n\nHayes, B. (1981)A Metrical Theory of Stress Rules.  Doctoral dissertation, MIT, Cambridge, \nMA. \n\nIt6, J.  (1989) A prosodic theory of epenthesis.  Natural Language and Linguistic Theory. \n7(2),217-259. \n\nLakoff,  G.  (1989)  Cognitive phonology.  Draft of paper presented  at the  UC-Berkeley \nWorkshop on Constraints vs.  Rules, May 1989. \n\nTouretzky, D. S., and Wheeler, D. W.  (1990a) A computational basis for phonology.  In D. \nS. Touretzky (ed.), Advances in Neural Information Processing Systems 2, 372-379.  San \nMateo, CA: Morgan Kaufmann. \n\nTouretzky, D.  S., and Wheeler, D.  W.  (1990b) Two derivations suffice:  the role of syllab(cid:173)\nification in cognitive phonology.  In C. Tenny (ed.), The MIT Parsing Volume, 1989-1990, \n21-35.  MIT Center for Cognitive SCience, Parsing Project Working Papers 3. \n\nTouretzky, D. S., and Wheeler, D. W. (1991) Sequence manipulation using parallel mapping \nnetworks. Neural Computation 3(1):98-109. \n\nWheeler, D. W., and Touretzky, D. S. (1991) From syllables to stress:  acognitivelyplausible \nmodel.  In  K.  Deaton,  M.  Noske, and M. Ziolkowski (eds.),  CLS 26-l/: Papers from  the \nP arasession on The Syllable in Phonetics and P hono logy, 1990. Chicago Linguistic Society. \n\n\f", "award": [], "sourceid": 370, "authors": [{"given_name": "David", "family_name": "Touretzky", "institution": null}, {"given_name": "Deirdre", "family_name": "Wheeler", "institution": null}]}