{"title": "An Integrated Architecture of Adaptive Neural Network Control for Dynamic Systems", "book": "Advances in Neural Information Processing Systems", "page_first": 1031, "page_last": 1038, "abstract": null, "full_text": "An Integrated Architecture of Adaptive Neural Network \n\nControl for Dynamic Systems \n\nRobert L. Tokar2 \n\nBrian D.McVey2 \n\n'Center for Nonlinear Studies, 2Applied Theoretical Physics Division \n\nLos Alamos National Laboratory, Los Alamos, NM, 87545 \n\nAbstract \n\nIn this study, an integrated neural network control architecture for nonlinear dynamic systems is \npresented. Most of the recent emphasis in the neural network control field has no error feedback as the \ncontrol input, which rises the lack of adaptation problem. The integrated architecture in this paper \ncombines feed forward control and error feedback adaptive control using neural networks. The paper \nreveals the different internal functionality of these two kinds of neural network controllers for certain \ninput styles, e.g., state feedback and error feedback. With error feedback, neural network controllers \nlearn the slopes or the gains with respect to the error feedback, producing an error driven adaptive \ncontrol systems. The results demonstrate that the two kinds of control scheme can be combined to \nrealize their individual advantages. Testing with disturbances added to the plant shows good tracking \nand adaptation with the integrated neural control architecture. \n\n1 INTRODUCTION \n\nNeural networks are used for control systems because of their capability to approximate nonlinear \nsystem dynamics. Most neural network control architectures originate from work presented by \nNarendra[I), Psaltis[2) and Lightbody[3) . In these architectures, an identification neural network is \ntrained to function as a model for the plant. Based on the neural network identification model, a neural \nnetwork controller is trained by backpropagating the error through the identification network. After \ntraining, the identification network is replaced by the real plant. As is illustrated in Figure 1, the \ncontroller receives external inputs as well as plant state feedback inputs. Training procedures are \nemployed such that the networks approximate feed forward control surfaces that are functions of \nexternal inputs and state feedbacks of the plant (or the identification network during training). \nIt is worth noting that in this architecture, the error between the plant output and the desired output of \nthe reference model is not fed back to the controller, after the training phase. In other words, this error \ninformation is ignored when the neural network applies its control. It is well known in control theory \nthat the error feedback plays a significant role in adaptation. Therefore, when model uncertainty or \nnoise/disturbances are present, a feed forward neural network controller with only state feedback will \nnot adaptively update the control signal. On line training for the neural controller has been proposed to \nobtaip adaptive ability[I)[3). However, the stability for the on line training of the neural network \ncontroller is unresolved[1][4]. \nIn this study, an additional nonlinear recurrent network is combined with the feed forward neural \nnetwork controller to form an adaptive controller. This added neural network uses feedback error \nbetween the reference model output and the plant output as an input In addition, the system's external \n\n\f1032 \n\nLiu Ke, Robert L. Tokar, Brian D. McVey \n\ninputs and the plant states are also input to the feedback network. This architecture is used in the control \ncommunity, but not with neural network components. The approach differs from a conventional error \nfeedback controller, such as a gain scheduled PID controller, in that the neural network error feedback \ncontroller implements a continuous nonlinear gain scheduled hypersurface, and after training, adaptive \nmodel reference control for nonlinear dynamic systems is achieved without further parameter \ncomputation. The approach is tested on well-known nonlinear control problems in the neural network \nliterature, and good results are obtained. \n\n2 NEURAL NETWORK CONTROL \n\nIn this section, several different neural network control architectures are presented. In these structures, \nidentification neural networks, viewed as accurate models for real plants, are used. \n\n2.1 NEURAL NETWORK FEED FORWARD CONTROL \n\nThe neural network controllers are trained by backpropagation of errors through a well trained neural \nidentification network. In this architecture, the state variable yet) of the system is sent back to the neural \nnetwork, and the external input x(t) also is input to the network. With these inputs, the neural network \nestabJishes a feed forward mapping from the external input x(t) to the control signal u(t). This control \nmapping is expressed as a function of the external input x(t) and the plant state yet): \n\nu(t)==j(x(t), yet\u00bb~ \n\n(1) \n\nwhere x(t)=[x(t), x(t-l), .. J, andy(t)=[y(t), y(t-l), . .Y. \nThis neural network control architecture is denoted in this study as feed forward neural control even \nthough it includes state feedback. Neural control with error feedback is denoted as feedback neural \ncontrol. \n\nx(t) \n-..:...r-----~Ref. Modelf-------, \n\nx(t) \n-~----~Ref. Modell-----.., \n\ne(t+ 1) \n\nControl NN \n\ny(t+ 1) \n\nu(t) ,-------, \n\nf----+-.. \n\ny(t+ 1) \n\nFigure I Neural Network Control Architecture. \nID NN represents the identification network. \nRef. Model means reference model, and NN \nmeans neural network. \n\nFigure 2 Neural Network Feedback Control \nArchitecture \n\nDuring the training phases, based on the assumption that the neural identification network provides a \nmodel for the plant, the gradient information needed for error backpropagation is obtained by calculating \nthe Jacobian of the identification network. The following equation describes this process for the control \narchitecture shown in Figure I. If the cost function is defined as E, then the gradient of the cost function \nwith respect to weight w of the neural controller is \n\n\fAn Integrated Architecture of Adaptive Neural Network Control for Dynamic Systems \n\n1033 \n\na E J a Yt-l \na E a E a u \na: = a; a w + a u a Yt-l + a Yt-l --a;-\n\n(a E a u \n\n(2) \n\nwhere u is tbe control signal and YI-1 is tbe plant feedback state. \nAfter tbe training stage, tbe neural network supplies a control law. Because neural networks have the \nability to approximate any arbitrary nonlinear functions[5], a feed forward neural network can build a \nnonlinear controller, which is crucial to tbe use of tbe neural network in control engineering. Also, since \nall tbe parameters of the neural network identification model and tbe neural network controller are \nobtained from learning through samples, matbematically untraceable features of tbe plant can be \nextracted from tbe samples and imbedded into tbe control system. \nHowever, because tbe feed forward controller has no error feedback, tbe controller can not adapt to tbe \ndisturbances occurring in tbe plant or tbe reference model. This problem is of substantial importance in \ntbe context of adaptive control. In tbe next subsection, error feedback between tbe reference models \nand tbe plant outputs is introduced into neural network controllers for adaptation. \n\n2.2 NEURAL ADAPTIVE CONTROL WITH ERROR FEEDBACK \n\nIt is known that feedback errors from the system are important for adaptation. Due to the flexibility of the \nneural network architecture, the error between the reference model and the plant can be sent back to the \ncontroller as an extra input. \nIn such an architecture, neural networks become nonlinear gain scheduled \ncontrollers with smooth continuous gains. Figure 2 shows the architecture for the feedback neural control. \nWith tbis architecture, tbe neural network control surface is not tbe fixed mapping from tbe x(t) to u(t) \nfor each state y(t), but instead it learns tbe slope or tbe gain referring to tbe feedback error e(t) for \ncontrol. This gain is a continuous nonlinear function of tbe external input x(t) and tbe state feedback \nyet). Figure 3 shows tbe recurrent network architecture of tbe feedback neural controller. The output \nnode needs to be recurrent because tbe output witbout tbe recurrent link from tbe neural controller is \nonly a correction to tbe old control signal, and tbe new control signal should be tbe combination of old \ncontrol signal and tbe correction. The otber nodes of tbe network can be feed forward or recurrent. If \nwe denote tbe weight for tbe output node's recurrent link as w., tben tbe output from tbe recurrent link is \nw.u(t-l). The following equation describes the feedback network. \n\nu(t) = wbu(t-I )+j(X(t), y(t), e(t\u00bb \n\n(3) \n\nwhere j(.) is a nonlinear function established by tbe network for which tbe recurrent link output is not \nincluded and e(t)=[e(t), e(t-I), ... f \nTo compare tbe control gain expression with conventional control theory, consider tbe Taylor series \nexpansion of tbe network forward mappingj(.), equation (3) becomes \n\nu(t) = w.u(t-l) + !'(x(t). yet\u00bb~ e(t)+ j\"(x(t), yet\u00bb~ e2(t)+... \n\n(4) \n\nwhere f'(x(t), y(t\u00bb=[ i1j(x(t), y(t), e(t\u00bb/ae(t), aJ!:x(t), y(t), e(t\u00bbIi1e(t-I), ... ]. \nignored and gO representsf'O, we get \n\nIf high order terms are \n\nu(t) = wbu(t-I)+ g(x(t), yet\u00bb~ e(t) \n\n(5) \n\n\f1034 \n\nLiu Ke, Robert L. Tokar, Brian D. McVey \n\nwhich is a gain scheduled controller and the gain is the function of external input x(/) and the plant state \nIt is clear that when w.=l.O, g(.) is a constant vector and e(/)=[e(t), e(t-l), e(t-2)]T, the feedback \ny(/). \nneural network controller degenerates to a discrete PID controUer. Because the neural network can \napproximate arbitrary nonlinear functions through learning, the neural network feedback controller can \ngenerate a nonlinear continuous gain hypersurface. \n\nRef. Model \n\nFigure 3 Feedback Neural Network Controller \n\nFigure 4 Integrated NN Control Architeture. \n\nIn the training process, error backpropagating through the identification network is used. The process is \nsimilar to the training of a feed forward neural controller, but the resulting control surface is completely \ndifferent due to the different inputs. After training, the neural network is able to provide a nonlinear \ncontrol law, \nthat is, the desired model following response can be obtained with fixed controller \nparameters for nonlinear dynamic systems. Traditionally, the control of the nonlinear plant is derived \nfrom continuous computing of the controller gains. \nThis feedback controller is error driven. As long as an error exists, \naccording to the error and the gain. This kind of neural controller is an adaptive controller in principle. \n\nthe control signal is updated \n\n2.3 INTEGRATED NEURAL NETWORK CONTROLLER \n\nThe characteristics of feed forward and error feedback neural control networks are described in the \nprevious subsections. In this section. the two controllers are combined. Figure 4 shows the architecture. \n\nIn this architecture, we include both feed forward and feedback neural network controllers. The control \nsignal is the combination from these two networks' outputs. In the training stage, it is our experience \nthat the feed forward network should be trained first. The feedback network is not included while \ntraining the feed forward network. After training the feed forward controller, the error feedback network \nis trained with the feed forward network, but the feed forward networks' weights are unchanged. \nBackpropagating the error through the identification network is applied for the training of both \nnetworks. \n\nWhen training the feedback control network, the feed forward calculation is \n\nu(t) = ujt)+u/b(t). \n\ny(t+ 1) = P(x(t), y(t), u(t\u00bb, \n\n(6) \n\n(7) \n\nwhere uj/) is the output from the feed forward controller network and u,..(t) is the output from the \nfeedback controller network, P(.) is the identification mapping. \n\n\fAn Integrated Architecture of Adaptive Neural Network Control for Dynamic Systems \n\n1035 \n\n3 CONTROL ON EXAMPLE PROBLEMS \n\nIn this section, the control architecture described above is applied to a well-known problem from the \nliterature[I). The plants and the reference model of the sample problems are described by difference \nequations \n\nplant: \n\ny(t + 1) = \n\nyet) \n2 \n\n1.0+ Y (t) \n\n+ (u(t) -1.O)u(t)(u(t) + 1.0) \n\nreference model: \n\ny(t + 1) = 0.6y(t) + u(t) \n\nThis is a nonlinear time varying dynamic system with no analytical inverse. \n\n3.1 FEED FORWARD CONTROL \n\n(II) \n\n(12) \n\nA feed forward neural network is trained to control the system to follow the reference model. The plant \nstate yet) and external inputx(t) are fed to the controller. During the training, the x(t) is randomly \ngenerated. After training, the controller generates a control signal u(t) such that the plant can follow the \nreference model output. Figure 5 shows the testing result of the reference model output and the \ncontrolled plant output. The input function is x(t)=sin(21ttf25)+sin(21tt/1O). The controller network \narchitecture is (2, 20, 1). \n\n4 \n\n2 \n\nOJ \n0 c \n\nQ) \n10-\nOJ \n\n~ 0 \n'\" 1J \n\nc \n0 \n'\" \n\n-2 \n\n20 \n\n40 \n\n60 \n\n80 \n\n100 \n\n2 \n\n::J \n0 \n~ \nc \n0 \nu \n\n-1 \n\n? \n(j \n~ \n\n0 \n() \n\n2 \n\n0 \n\n..... 1 \n\n..... 2 \n\nFigure 5 Tracking Result From the Feed Forward NN. \nOutput of reference (solid line) and plant (dash line). \n\nFigure 6 Feed Forward Control Surface \n\nThe output surface of the controller network is shown in Figure 6. By examining the controller output \nsurface, we can see that the neural network builds a feed forward mapping from x(t) to u(t). This feed \nforward mapping is also a function of the plant state yet). Under each state, the neural network \ncontroller accepts input x(t) to produce control signal u(t) such that the plant follows the reference model \nreasonably well. In Figure 6, the x axis is the external input x(t) and the y axis is the plant feedback \noutput yet). The z axis represents the control surface. \nThe feed forward controller laCks the ability to adapt to plant uncertainty, noise or changes in the \nreference model. As an example, we apply this feed forward controller to the disturbed plant with a bias \n0.5 added to the original plant. The tracking result is shown in Figure 7. With this slight bias, the plant \ndoes not follow the reference model. Clearly, the feed forward controller has no adaptive ability to this \nmodel bias. \n\n\f1036 \n\nLiu Ke, Robert L. Tokar, Brian D. McVey \n\n3.2 FEEDBACK CONTROL \n\nFtrSt, we compare the neural network feedback controller with fixed gain PID controllers. For many \nnonlinear systems, the fixed gain PID controllers will give poor tracking and continuous adaptation of \nthe controller parameters is needed. The neural network approach offers an alternative control approach \nfor nonlinear systems. Through the training, control gains, imbedded in the neural network, are \nestablished as a continuous function of system external inputs x(t) and plant states yet). \n\nThe sample problem in the above section is now employed to describe how the neural network creates a \nnonlinear control gain surface with error feedback and additional inputs. First, we show one simple case \nof neural adaptive feedback controller. This controller can only adapt to the system nonlinearity with a \nfixed linear input pattern. The reason to show this simple adaptation case first is that its control gain \nsurface can be illustrated graphically. \nFigure 8 illustrates, for the system in equations (11) and (12) that a fixed gain PI controller fails to track \nthe reference model, for even one fixed linear input pattern x(t)=0.2t-2.5, because the plant nonlinearity. \nFigure 9 illustrates the result from a recurrent neural network with feedback error e(t) and x(t) as inputs. \nThe neural network is trained by backpropagation error through the identification network. Compared to \nthe flXed gain PI controller, the neural network improves the tracking ability significantly. \n\n., \n.. ., \n0 c ., \n'!! \n\n>. \n-.::I \nC \n0 \n>. \n\n4 \n\n2 \n\n0 \n\n-2 \n\n6 \n\n3 \n\n0 \n\n- 3 \n\nOJ \n. \n'0 \nc \n0 \n>. \n\n, \n\nI \nI \n\n20 \n\n40 \n\nt \n\n60 \n\n80 \n\n100 \n\no \n\n5 \n\n10 15 20 25 30 35 \n\nt \n\nFigure 7 Tracking Result for Shifted Plant, plant \noutput (dash line) and reference output (solid line). \n\nFigure 8 Reference Model Output (solid line) \nand PID Controlled Plant Output (dashed line) \n\nThe control surface of the updating output fl.) is shown in Figure 10, which is the output from the neural \nnetwork controller without recurrent link (see equation (3\u00bb. We plot the surface of the updating output \nfrom the controller with respect to input x(t) and error feed back input e(t). The gain of the controller is \nequivalent to the updating output from the network when error=l.O. As shown in the figure, the gain in \nthe neighborhood about x(t)=O changes largely according to the direction of changes in the plant in the \ncorresponding region. The updating surface for a PID controller is a plane. The neural network \nimplements a nonlinear continuous control gain surface. \n\nFor a more complicated case, we addx(t-I) as another input to the neural network as well as e(t-l), and \ntrain by error backpropagation through the identification network. These two inputs, x(t) and x(t-I) add \ndifference information to the network. The network can adapt to not only different operating regions \nindicated by x(t), but also different input patterns. Figure 11 shows the tracking results with two \ndifferent input patterns. In Figure II (a), input pattern is x(t)=4.0sin(tI4.0). \nIn Figure 11 (b) input \npattern is x(t)=sin(21t1!25)+sin(21t111O). \n\n\fAn Integrated Architecture of Adaptive Neural Network Control for Dynamic Systems \n\n1037 \n\n6 \n\n3 \n\nQ) \nu \nc \n~ \n2 \n, \n~ \n, \n>. \n-0. -3 I \nc \n0 \n>. - 6 \n\n0 \n\nI \n\no \n\n5 \n\n1 0 \n\n1 5 20 25 30 35 \n\nt \n\nFigure 9 Reference Model Output (solid line) and \nNeural Network Controled Output (dashed line) \n\nFigure 10 Feedback Neural Controller Updating Surface \n\nOJ \nu \nc \n~ \n~ \n~ \n>. \n\n0 \n- 2 \n\" \nc: \n- 4 \n0 \n>. -6 \n- 8 \n-10 \n0 \n\n20 \n\n40 \n(a) \n\n60 \n\n80 \n\n10C \n\n., \n\n5 \n4 \n3 \nu \nc \n2 \n~ \n1 \nOJ \nl' \n0 \n>. -1 \n\" - 2 \nc \n- 3 \n0 \n>. - 4 \n-5 \n-6 \n0 \n\n20 \n\n40 \n\n60 \n\n80 \n\n100 \n\n(b) \n\nFigure 11 Output of the Reference Model (solid line) and the Plant (dash line) \n\n3.3 INTEGRA TED NEURAL CONTROLLER \n\nAs shown in the above section, when only error feedback neural controller is used, the control result is \nnot very accurate. Now we combine feed forward and feedback control to realize good tracking and \nadaptation. Figure 12 shows the control result from the integrated controller when the plant is shifted \nO.S. Compared to only feed forward control (Figure 7), the integrated controller has much better \nadaptation to the shifted plant. \nWhen the plant changes, adding an extra feed back controller can avoid on-line training of feed forward \nnetwork which may induce potential instability, and the adaptation is achieved. The output from the \nfeedback network controller is driven by the error between the reference model and the plant. \n\n4 DISCUSSIONS \n\nWe have emphasized in the above sections that a feed forward controller with only state feedback does \nnot adapt when model uncertainties or noise/disturbance are present. The presence of a feed back \ncontroller can make the on line training of the feed forward network unnecessary, thus avoiding \npotential instability. The main reason for the instability of on-line training is the incompleteness of \nsample sets, which is referred to as a lack of persistent excitation in control theory[6]. First, it leads to \nan inaccurate identification network. Training with this network can result in an unstable controller. \nSecond, it makes the training of controller away from global representation. With an error feedback \nadaptive network, the output from the feedback network controller is driven by the error between the \nreference model and the plant. In the simplest case when all the activity functions are linear and only \nthe feedback errors are inputs, this kind of neural network is equivalent to a PID controller. However, \n\n\f1038 \n\nLiu Ke, Robert L Tokar, Brian D. McVey \n\nbeyond the scope of PID controllers, the neural networks are capable to approximating nonlinear time \nvariant control gain surfaces corresponding to different operating regions. Also, unlike a PID controller, \nthe coefficients for the neural adaptive controller are obtained through a training procedure. \n\n4 \n\nQ) \n\nu c \n1:' \n0; ... \n\nQ) \n\n>, \n\"0 \nC \n0 \n>, \n\no \n\n20 \n\n40 \n\n60 \n\n80 \n\n100 \n\nFigure 12 Integrated Network Controller Tracking Result for Shifted Plant. \n\nPlant Output (dash line) and Reference Output (solid line). \n\nIt has rise time, overshoot \nThe error feedback network behaves as a gain scheduling controller. \nconsideration and delay problem. Feed forward control can compensate for these problems to some \ndegree. For example, the feed forward network can perform a nonlinear mapping with designed time \ndelay. Therefore with the feed forward network, the delay problem maybe overcame significantly. Also \nthe feed forward controller can help to reduce rise time compare to use only feedback controller. \n\nWith the feed forward network, the feedback network controller can have much smaller gains compared \nto using a feedback network alone. This increases the noise rejection ability. Also this reduces the \novershoot as well as settle time. \n\nThe neural network control architecture offers an alternative to the conventional approach. It gives a \ngeneric model for the broadest class of systems considered in control theory . However this model needs \nto be configured depending on the details of the control problem. With different inputs, the neural \nnetwork controllers establish different internal hyperstates. When plant states are fed back to the \nnetwork, a feed forward mapping is established as a function of the plant states by the neural network \ncontroller. When the errors between the reference model and the plant are used as the error feedback \ninputs to a dynamic neural network controller, \nthe network functions as an associative memory \nnonlinear gain scheduled controller. The above two kinds of neural controller can be combined and \ncomplemented to achieve accurate tracking and adaptation. \n\nReferences \n\n[1] Kumpati S. Narendra and Kannan Parthasarathy. \"Gradient Methods for the Optimization of DynamiCal \n\nSystems Containing Neural Networks,\" IEEE Trans. Neural Networks. vol. 2. pp252-262 Mar. 1991 \n\n[2] Psaltis. D .\u2022 Sideris. A. and Yamamura. A., \"Neural controllers.\" Proc. of 1st International Conference on \n\nNeural Networks. Vol. 4. pp551-558. San Diego. CA. 1987 \n\n[3) G. lightbOdy. Q. H. Wu and G. W. Irwin. \"Control applications for feed forward networks.\" Chapter 4. \n\nNeural Networks for Control and Systems. Edited by K.warwich, G. W. Irwin and K. J. Hunt 1992 \n\n[4) R. Abikowski and P. 1. Gawthrop. \"A survey of neural networks for control\" Chapter 3. NeUral Networks \nfor Control and Systems. ISBN 0-86341-279-3. Edited by K.warwich. G. W. Irwin and K. 1. Hunt 1992 \n[5] John Hertz. Anders Krogh and Richard G. Palmer. \"Introduction to the Theory of Neural Computation.\" \n[6J Thomas Miller. RiChard S. Sutton and Paul 1. Werbos. \"Neural Networks for Control\" \n\n\f", "award": [], "sourceid": 967, "authors": [{"given_name": "Ke", "family_name": "Liu", "institution": null}, {"given_name": "Robert", "family_name": "Tokar", "institution": null}, {"given_name": "Brain", "family_name": "McVey", "institution": null}]}