{"title": "Improvisation and Learning", "book": "Advances in Neural Information Processing Systems", "page_first": 1377, "page_last": 1384, "abstract": "", "full_text": "Improvisation and Learning\n\nJudy A. Franklin \n\nComputer Science Department\n\nSmith College\n\nNorthampton, MA 01063\njfranklin@cs.smith.edu\n\nAbstract\n\nThis article presents a 2-phase computational learning model and appli-\ncation. As a demonstration, a system has been built, called CHIME for\nComputer Human Interacting Musical Entity. In phase 1 of training, re-\ncurrent back-propagationtrains the machine to reproduce 3 jazz melodies.\nThe recurrent network is expanded and is further trained in phase 2 with a\nreinforcement learning algorithm and a critique produced by a set of basic\nrules for jazz improvisation. After each phase CHIME can interactively\nimprovise with a human in real time.\n\n1 Foundations\n\nJazz improvisation is the creation of a jazz melody in real time. Charlie Parker, Dizzy Gille-\nspie, Miles Davis, John Coltrane, Charles Mingus, Thelonious Monk, and Sonny Rollins et\nal. were the founders of bebop and post bop jazz [9] where drummers, bassists, and pianists\nkeep the beat and maintain harmonic structure. Other players improvise over this structure\nand even take turns improvising for 4 bars at a time. This is called trading fours.\n\nMeanwhile, arti\ufb01cial neural networks have been used in computer music [4, 12]. In partic-\nular, the work of (Todd [11]) is the basis for phase 1 of CHIME, a novice machine improvi-\nsor that learns to trade fours. Firstly, a recurrent network is trained with back-propagation\nto play three jazz melodies by Sonny Rollins [1], as described in Section 2. Phase 2 uses\nactor-critic reinforcement learning and is described in Section 3. This section is on jazz ba-\nsics.\n\n1.1 Basics: Chords, the ii-V-I Chord Progression and Scales\n\nThe harmonic structure mentioned above is a series of chords that may be reprated and that\nare often grouped into standard subsequences. A chord is a group of notes played simulta-\nneously. In the chromatic scale, C-Db-D-Eb-E-F-Gb-G-Ab-A-Bb-B-C, notes are separated\nby a half step. A \ufb02at (b) note is a half step below the original note; a sharp (#) is a half above.\nTwo half steps are a whole step. Two whole steps are a major third. Three half steps are a\nminor third.\n\nA major triad (chord) is the \ufb01rst or tonic note, then the note a major third up, then the note a\nminor third up. When F is the tonic, F major triad is F-A-C. A minor triad (chord) is the tonic\n\n\u0001 www.cs.smith.edu/\u02dcjfrankli\n\n\fthen a minor third, then a major third. F minor triad is F-Ab-C. The diminished triad is the\ntonic, then a minor third, then a minor third. F diminished triad is F-Ab-Cb. An augmented\ntriad is the tonic, then a major third, then a major third. The F augmented triad is F-A-Db.\n\nA third added to the top of a triad forms a seventh chord. A major triad plus a major third is\nthe major seventh chord. F-A-C-E is the F major seventh chord (Fmaj7). A minor triad plus\na minor third is a minor seventh chord. For F it is F-Ab-C-Eb (Fm7). A major triad plus a\nminor third is a dominant seventh chord. For F it is F-A-C-Eb (F7). These three types of\nchords are used heavily in jazz harmony. Notice that each note in the chromatic scales can\nbe the tonic note for any of these types of chords.\n\nA scale, a subset of the chromatic scale, is characterized by note intervals. Let W be a whole\nstep and H be a half. The chromatic scale is HHHHHHHHHHHH. The major scale or ionian\nmode is WWHWWWH. F major scale is F-G-A-Bb-C-D-E-F. The notes in a scale are de-\ngrees; E is the seventh degree of F major. The \ufb01rst, third, \ufb01fth, and seventh notes of a major\nscale are the major seventh chord. The \ufb01rst, third, \ufb01fth, and seventh notes of other modes\nproduce the minor seventh and dominant seventh chords. Roman numerals represent scale\ndegrees and their seventh chords. Upper case implies major or dominant seventh and lower\ncase implies minor seventh [9]. The major seventh chord starting at the scale tonic is the I\n(one) chord. G is the second degree of F major, and G-Bb-D-F is Gm7, the ii chord, with re-\nspect to F. The ii-V-I progression is prevalent in jazz [9], and for F it is Gm7-C7-Fmaj7. The\nminor ii-V-i progression is obtained using diminished and augmented triads, their seventh\nchords, and the aeolian mode. Seventh chords can be extended by adding major or minor\nthirds, e.g. Fmaj9, Fmaj11, Fmaj13, Gm9, Gm11, and Gm13. Any extension can be raised\nor lowered by 1 step [9] to obtain, e.g. Fmaj7#11, C7#9, C7b9, C7#11.\n\nMost jazz compositions are either the 12 bar blues or sectional forms (e.g. ABAB, ABAC,\nor AABA) [8]. The 3 Rollins songs are 12 bar blues. \u201cBlue 7\u201d has a simple blues form. In\n\u201cSolid\u201d and \u201cTenor Madness\u201d, Rollins adds bebop variations to the blues form [1]. ii-V-I\nand VI-II-V-I progressions are added and G7+9 substitutes for the VI and F7+9 for the V\n(see section 1.2 below); the II-V in the last bar provides the turnaround to the I of the \ufb01rst bar\nto foster smooth repetition of the form. The result is at left and in Roman numeral notation\n\nI\n\nI\nI\nIV IV I\nii\n\nV\n\nI\nVI\nII V\n\nI VI\n\nat right:\n\nBb7\nEb7\nCm7\n\nBb7 Bb7\nEb7 Bb7\nF7\n\nBb7 G7+9 C7 F7+9\n\nBb7\nG7+9\n\n1.2 Scale Substitutions and Rules for Reinforcement Learning\n\nFirst note that the theory and rules derived in this subsection are used in Phase 2, to be de-\nscribed in Section 3. They are presented here since they derive from the jazz basics immedi-\nately preceding. One way a novice improvisor can play is to associate one scale with each\nchord and choose notes from that scale when the chord is presented in the musical score.\nTherefore, Rule 1 is that an improvisor may choose notes from a \u201cstandard\u201d scale associ-\nated with a chord. Next, the 4th degree of the scale is often avoided on a major or dominant\nseventh chord (Rule 3), unless the player can resolve its dissonance. The major 7th is an\navoid note on a dominant seventh chord (Rule 4) since a dominant seventh chord and its\nscale contain the \ufb02at 7th, not the major 7th.\n\nRule 2 contains many notes that can be added. A brief rationale is given next. The C7 in\nGm7-C7-Fmaj7 may be replaced by a C7#11, a C7+ chord, or a C7b9b5 or C7alt chord [9].\nThe scales for C7+ and C7#11 make available the raised fourth (\ufb02at 5), and \ufb02at 6 (\ufb02at 13)\nfor improvising. The C7b9b5 and C7alt (C7+9) chords and their scales make available the\n\ufb02at9, raised 9, \ufb02at5 and raised 5 [1]. These substitutions provide the notes of Rule 2. These\nrules (used in phase 2) are stated below, using for reinforcement values very bad (-1.0), bad\n(-0.5), a little bad (-0.25), ok (0.25), good (0.5), and very good (1.0). The rules are discussed\n\n\ffurther in Section 4.\nThe Rule Set:\n1) Any note in the scale associated with the chord is ok (except as noted in rule 3).\n2) On a dominant seventh, hip notes 9, \ufb02at9, #9, #11, 13 or \ufb02at13 are very good. One hip\nnote 2 times in a row is a little bad. 2 hip notes more than 2 times in a row is a little bad.\n3) If the chord is a dominant seventh chord, a natural 4th note is bad.\n4) If the chord is a dominant seventh chord, a natural 7th is very bad.\n5) A rest is good unless it is held for more than 2 16th notes and then it is very bad.\n6) Any note played longer than 1 beat (4 16th notes) is very bad.\n7) If two consecutive notes match the human\u2019s, that is good.\n\n2 CHIME Phase 1\n\nIn Phase 1, supervised learning is used to train a recurrent network to reproduce the three\nSonny Rollins melodies.\n\n2.1 Network Details and Training\n\nThe recurrent network\u2019s output units are linear. The hidden units are nonlinear (logistic\nfunction). Todd [11] used a Jordan recurrent network [6] for classical melody learning and\ngeneration. In CHIME, a Jordan net is also used, with the addition of the chord as input\n(Figure 1. 24 of the 26 outputs are notes (2 chromatic octaves), the 25th is a rest, and the\n26th indicates a new note. The output with the highest value above a threshold is the next\nnote, including the rest output. The new note output indicates if this is a new note, or if it\nis the same note being held for another time step ( \u0002\u0001\u0004\u0003\u0006\u0005 note resolution).\n\nThe 12 chord inputs (12 notes in a chromatic scale), are 1 or 0. A chord is represented as\nits \ufb01rst, third, \ufb01fth, and seventh notes and it \u201cwraps around\u201d within the 12 inputs. E.g., the\nFm7 chord F-Ab-C-Eb is represented as C, Eb, F, Ab or 100101001000. One plan input per\nsong enables distinguishing between songs. The 26 context inputs use eligibility traces, giv-\ning the hidden units a decaying history of notes played. CHIME (as did Todd) uses teacher\nforcing [13], wherein the target outputs for the previous step are used as inputs (so erro-\nneous outputs are not used as inputs). Todd used from 8 to 15 hidden units; CHIME uses\n50. The learning rate is 0.075 (Todd used 0.05). The eligibility rate is 0.9 (Todd used 0.8).\nDifferences in values perhaps re\ufb02ect contrasting styles of the songs and available computing\npower.\n\nTodd used 15 output units and assumed a rest when all note units are \u201cturned off.\u201d CHIME\nuses 24 output note units (2 octaves). Long rests in the Rollins tunes require a dedicated\noutput unit for a rest. Without it, the note outputs learned to turn off all the time. Below\nare results of four representative experiments. In all experiments, 15,000 presentations of\nthe songs were made. Each song has 192 16th note events. All songs are played at a \ufb01xed\ntempo. Weights are initialized to small random values. The squared error is the average\nsquared error over one complete presentation of the song. \u201cFinessing\u201d the network may\nimprove these values. The songs are easily recognized however, and an exact match could\nimpair the network\u2019s ability to improvise. Figure 2 shows the results for \u201cSolid.\u201d\nExperiment 1. Song: Blue Seven. Squared error starts at 185, decreases to 2.67.\nExperiment 2. Song: Tenor Madness. Squared error starts at 218, decreases to 1.05.\nExperiment 3. Song: Solid. Squared error starts at 184, decreases to 3.77.\nExperiment 4. Song: All three songs: Squared error starts at 185, decreases to 36.\n\n\fFigure 1: Jordan recurrent net with addition of chord input\n\n2.2 Phase 1 Human Computer Interaction in Real Time\n\nIn trading fours with the trained network, human note events are brought in via the MIDI\ninterface [7]. Four bars of human notes are recorded then given, one note event at a time\nto the context inputs (replacing the recurrent inputs). The plan inputs are all 1. The chord\ninputs follow the \u201cSolid\u201d form. The machine generates its four bars and they are played in\nreal time. Then the human plays again, etc. An accompaniment (drums, bass, and piano),\nproduced by Band-in-a-Box software (PG Music), keeps the beat and provides chords for\nthe human.\n\nFigure 3 shows an interaction. The machine\u2019s improvisations are in the second and fourth\nlines. In bar 5 the \ufb02at 9 of the Eb7 appears; the E. This note is used on the Eb7 and Bb7\nchords by Rollins in \u201cBlue 7\u201d, as a \u201cpassing tone.\u201d D is played in bar 5 on the Eb7. D is the\nnatural 7 over Eb7 (with its \ufb02at 7) but is a note that Rollins uses heavily in all three songs,\nand once over the Eb7. It may be a response to the rest and the Bb played by the human in\nbar 1. D follows both a rest and a Bb in many places in \u201cTenor Madness\u201d and \u201cSolid.\u201d In\nbar 6, the long G and the Ab (the third then fourth of Eb7) \ufb01gure prominently in \u201cSolid.\u201d At\nthe beginning of bar 7 is the 2-note sequence Ab-E that appears in exactly the same place\nin the song \u201cBlue 7.\u201d The focus of bars 7 and 8 is jumping between the 3rd and 4th of Bb7.\nAt the end of bar 8 the machine plays the \ufb02at 9 (Ab) then the \ufb02at 3 (Bb), of G7+9. In bars\n13-16 the tones are longer, as are the human\u2019s in bars 9-12. The tones are the 5th, the root,\nthe 3rd, the root, the \ufb02at 7, the 3rd, the 7th, and the raised fourth. Except for the last 2, these\nare chord tones.\n\n3 CHIME Phase 2\n\nIn Phase 2, the network is expanded and trained by reinforcement learning to improvise\naccording to the rules of Section 1.2 and using its knowledge of the Sonny Rollins songs.\n\n3.1 The Expanded Network\n\nFigure 4 shows the phase 2 network. The same inputs plus 26 human inputs brings the total\nto 68. The weights obtained in phase 1 initialize this network. The plan and chord weights\n\n\fFigure 2: At left \u201cSolid\u201d played by a human; at right the song reproduced by the ANN.\n\nare the same. The weights connecting context units to the hidden layer are halved. The same\nweights, halved, connect the 26 human inputs to the hidden layer. Each output unit gets the\n100 hidden units\u2019 outputs as input. The original 50 weights are halved and used as initial\nvalues of the two sets of 50 hidden unit weights to the output unit.\n\n3.2 SSR and Critic Algorithms\n\nUsing actor-critic reinforcement learning ([2, 10, 13]), the actor chooses the next note to\nfrom the critique made by the\nplay. The critic receives a \u201craw\u201d reinforcement signal\n. A\nrules of Section 1.2. For output j, the SSR (actor) computes mean\nGaussian distribution with mean\nis\ngenerated, the critic modi\ufb01es\nis further modi\ufb01ed by a self-scaling\nalgorithm that tracks, via moving average, the maximum and minimum reinforcement and\nuses them to scale the signal to produce\n\n\f\u000b\u000e\r\u0010\u000f\nchooses the output\n\nand standard deviation\n\n\u0001\u0010\u0002\nand produces\n\n\u0001\u0003\u0002\u0005\u0004\u0007\u0006\t\b\n\n\u0002\u0012\u0011\n\u0015\u0016\u0002\n\n\u0013\u0014\u0002\n\n\u0016\u0017\n\n\u0018\u0017\n\n.\n\n.\n\n.\n\n, then\n\n\u0004G\u0018%'&4\u0011\n\n)7\u00048%'&4\u0011\u00109\u001c\u001c%'&4\u0011\u0003\u001f-5:)3;:\n\u0017\u0012<\n)\u00106A\u001f\n)7\u0004A@B\u0018%2&4\u0011 \u001f!5\u00036\n, then\n\n\u0018%2&4\u0011 \u001f!5\u00036\n\",@C)D\n\u0018%2&4\u0011 \u001f!5\u00036\nThe goal is to make small gains in reinforcement more noticeable and to scale the values\nbetween -1 and 1. If\nand if\n, the extremes of -1 and\n:\n\n\u001a\u0004\t\u001b\u001c\u0011\u001e\u001d \u001f!\u0016\u0017#\"$\u0018%'&(\u0011*)+\",\u001b\u001c\u0011\u001e\u001d\u0003\u001f-\u0018%/.102\"$\u0018\u00173)\n\u0018%/.10=\u001f-5\u00106\n)=\u0004\t%2.>0+9\u001c\u0018%/.10=\u001f!5:)?;:\n\u00173<\n\u0018%/.10=\u001f-5\u00106\n)\u00106A\u001f\n\",@C)D\n)=\u0004E@B\u001c%2.10=\u001f!5F6\n\"A\u001b\u001c\u0011\u001e\u001d \u001f!\u0018%2.>0H\"\t\u0018%2&4\u0011*)I\u0004\n\"AJ\n\t\u0004\n . If\n\u001a\u00048\u001b\u001c\u0011\u001e\u001d \u001f!\u0018%2.>02\"I\u0018%'&4\u0011K)+\"\n\u0004\tJ7\"\n\u00024\u001f!5:)F6SRT\u0019\n\u0014\u001f!\u0015\u0018\u0002M\"U\u0001K\u0002\u001c)DVK\u0001K\u0002\u0018WXVK\u000f\n\u001f!5:)\u00036\t\u001f\n\nJMLNLHO\n1 are approached. The weight and standard deviation updates use\n\nIf the difference between the max and min reinforcement stays large, over time\ncrease (to a max of\ntime\nwill shrink (to a min of\npropagation as before, using\nf`T\u001f!\u0015\nSSR algorithm and its precursors.\n\nwill in-\n) and allow more exploration. When rmax-rmin is small, over\n). The actor\u2019s hidden units are updated using back-\nas \u201cerror.\u201d See [3, 5, 13] for more details on the\n\"g\u0001\n\n)=\u0004Q\u000f\n)=\u00048%'&4\u0011\u00109\u001c\u0013CY[Z3\\\u0014;]%/.10+9_^\u000e\u0013\n\n\"I^\u0010)\u0010`a\u001f!\u0018%'&4\u0011b\"U\u001c%2.10F)3;c\u0013BY\n\n\u00048\u0018%2.10\n\u0002P\u001f!5F6\n\u001f-5F6\n\n;cOdLH^IL\n\nY[Z3\\\n\n\u0013e\u0002\n\n<\u001e<\n\nThe critic inputs are the outputs of the hidden layer of the actor network; it \u201cpiggy-backs\u201d on\nthe actor and uses its learned features (see Figure 5). This also alleviates the computational\nburden so it can run in real time. There are delays in reward, e.g. in that a note played too\nmany times in a row may result in punishment, and if 2 notes in a row coincide with the\nhuman\u2019s it is rewarded. If\nis the prediction of future reward [10] for state x at time\nt,\n\n\u001d\u0003\u001f!\u0011\u0003;:5:)\n\n\n\n\n\n\u0019\n\n\n\n\n\n\u0017\n\n\n\n\u0017\n\n\u0017\n\u0019\n\n\n\n\u0017\n\u0019\n\n\u0019\n\n\u000f\n\n\n\n\u0002\n\u0013\n\u0002\n\n\u0002\n\n\n\b\n\n\u0013\n\u0002\n\u0013\n\u0013\nY\n\n\b\n\u0019\n\u0002\n\u0002\n)\n\fFigure 3: Phase1 trading 4 bars: 4 human, 4 machine, 4 human, 4 machine\n\n\u001f-5\u00106\n\nThe critic is a linear function of its inputs:\nupdated incrementally using the value of\n\n)=\u0004\te\u001f!5:)F6,^\n\n`\u0003\u001d \u001f!\u0011\u0003\u001f-5F6\n\n)3;:5:) \"2\u001d\u0003\u001f-\u0011\u0003\u001f-5:)3;:5:)\n\nfor\n\nLH^\n\n\f\u000b\u000e\n\n .\n\n\u001f!5:)D\u0011\n\n\u001f!5:)\n\nThe weights are\n\n\u001d \u001f!\u0011\u0003\u001f-5:)3;]5:)T\u0004\n:\n\n\u0016\u0017\n\n\u001f-5:)F6\n\n)=\u0004Q\u000f\n\nV4\u001d*W\u0016VK\u000f\n\n\u001c\u0017\n\u001f-5F6\nis in effect an error signal, a difference between consecutive predicted rewards [10]. The\ncritic also uses eligibility traces of the inputs, so\nwhere\n. While this is all experimental, initial results show that the system\nwith both the self-scalar and the critic performs better than with just one or without either\none. A more systematic study is planned.\n\nis actually\n\n\u0011 \u001f!5:)a\u0004\n\n\u0011 \u001f!5:)76\n\n\u001f-5:)\u001a\u0004\n\n\u001b\u0004\u0003\n\n\u0011 \u001f!5:)\n\n\u0011\u0003\u001f!5:)\n\n\u001f!5=\"\n\n4 Results and Comments\n\nRecall the rules of Section 1.2. Rules 1-4 are based on discussions with John Payne, a pro-\nfessional jazz musician and instructor of 25 years1. The rules by no means encompass all\nof jazz theory or practice but are a starting point. The notes in rule 2 were cast as good in\na \u201chip\u201d situation. The notion of hip requires human sophistication so for now these notes\nare reinforced if played sporadically on the dominant seventh. Rule 5 was added to dis-\ncourage not playing any notes. Rules 5 and 6 focus on not allowing an output of one note\n,\nfor too long. Each chord is assigned a scale for rule 1.\nO\u0006\u0005\f\u000b\nproviding stability, and deliberate action uncertainty so different notes are played, for the\nsame network state. Generally the goal of reinforcement learning is to \ufb01nd the best action\nfor a given state, with uncertainty used for further exploration. Here, reinforcement learning\n\ufb01nds the best set of actions for a given state. In a typical example using the phase 1 network\nprior to phase 2 improvement, the average reinforcement value according to the rule set is\n-.37 (on a scale from -1 to 1). After Phase 2, the average reinforcement value is .28 after\n30-100 off-line presentations of the human solo of 1800 note events.\n\nis limited,\n\nO\b\u0007\n\t$L\n\n\u0013e\u0002gL\n\n\u0013C\u0002\n\nO\u0006\u0005\n\nFigure 6 shows 12 bars of a human solo and 12 bars of a machine solo. The note durations\n\n1The rules are not meant to represent John Payne\n\n\n\u0017\n\n\u0017\n\nO\n\u0017\nL\n\u0006\nY\n\u000f\n\n\u000f\n\n\n\n\u0001\n\n\n\u0017\n\u0002\n\u001b\n\\\n\n)\n\u001b\n\\\n\u0002\n\fFigure 4: Recurrent reinforcement learning network with human input used in phase 2.\n\nFigure 5: Phase 2 network with critic \u201cpiggy-backing\u201d on hidden layer.\n\nare shortened, re\ufb02ecting the rules to prevent settling onto one note. The machine plays chord\ntones, such as Bb and D in bars 1 and 2. The high G is the 13 of Bb7, a hip note. In bars 3\nand 4 it plays C sharp, a hip note (the #9 of Bb7) and high G. These notes are played in bars\n9 and 11 on Bb7. In bars 5 and 6 the 9 and 13 (F and C) are played on Eb7. The natural 7\n(D) re\ufb02ects its heavy use in Rollins\u2019 melodies. Hip notes show up in bar 9 on Cm7: the 13\n(G) and the 9 (D). In bar 11 G is played on G7+9 as is the hip \ufb02at 9 (the Ab). In bar 12, the\nEb (\ufb02at 7 chord tone) is played on the F7+9. In bars 2, 4, 7, 9, and 10 the machine starts at\nthe G at the top of the staff and descends through several chord tones, producing a recurring\nmotif, an artifact of a \u201cgood\u201d jazz solo. The phase 2 network has been used to interact with a\nhuman in real time while still learning. It keeps its recurrence since the human has a separate\nset of inputs.\n\nA limitation to be addressed for CHIME is to move beyond one chord at a time. To achieve\nthis, it must use more context, over more time. There are plenty of improvisation rules for\nchord progressions [8]. Because CHIME employs reinforcement learning, it has a stochastic\nelement that allows it to play \u201coutside the chord changes.\u201d A research topic is to understand\nhow to enable it to do this more pointedly.\n\n\fFigure 6: At left, 12 bars of human solo. At right, 12 bars the machine plays in response.\n\nReferences\n\n[1] J. Aebersold. You can play Sonny Rollins. A New Approach to Jazz Improvisation Vol 8. Jamey\n\nAebersold, New Albany, IND., 1976.\n\n[2] A. G. Barto, R. S. Sutton, and C. W. Anderson. Neuronlike adaptive element that can solve dif-\n\ufb01cult learning control problems. IEEE Transactions on Systems, Man, and Cybernetics, SMC-\n13:834\u2013846, 1983.\n\n[3] H. Benbrahim and J. Franklin. Biped walking using reinforcement learning. Robotics and Au-\n\ntonomous Systems, 22:283\u2013302, 1997.\n\n[4] N. Grif\ufb01th and P. Todd. Musical Networks: Parallel Distributed Perception and Performance.\n\nMIT Press, Cambridge MA, 1999.\n\n[5] V. Gullapalli, J. Franklin, and H. Benbrahim. Acquiring robot skills via reinforcement learning.\n\nIEEE Control Systems Magazine, 1994.\n\n[6] M. Jordan. Attractor dynamics and parallelism in a connectionist sequential machine. InPro-\n\nceedings of the Eighth Annual Conference of the Cognitive Science Society, 1986.\n\n[7] P. Messick. Maximum MIDI. Manning Publications, Greenwich, CT, 1988.\n[8] S. Reeves. Creative Jazz Improvisation. 2nd Ed. Prentice Hall, Upper Saddle River NJ, 1995.\n[9] M. A. Sabatella. Whole Approach to Jazz Improvisation. A.D.G. Productions, Lawndale CA,\n\n1996.\n\n[10] R. Sutton. Learning to predict by the methods of temporal differences. Machine Learning, 3:9\u2013\n\n44, 1988.\n\n[11] P. M. Todd. A connectionist approach to algorithmic composition. In P. M. Todd and e. D. Loy,\n\neditors, Music and Connectionism. MIT Press, Cambridge MA, 1991.\n\n[12] P. M. Todd and e. D. Loy. Music and Connectionism. MIT Press, Cambridge, MA, 1991.\n[13] R. J. Williams. Simple statistical gradient-following algorithms for connectionist reinforcement\n\nlearning. Machine Learning, 8:229\u2013256, 1992.\n\n\f", "award": [], "sourceid": 2079, "authors": [{"given_name": "Judy", "family_name": "Franklin", "institution": null}]}