F L Lewis Fellow IEEE Fellow U K

F. L. Lewis, Fellow IEEE, Fellow U. K. Inst. Meas. & Control Moncrief-O’Donnell Endowed Chair Head, Controls & Sensors Group Automation & Robotics Research Institute (ARRI) The University of Texas at Arlington http: //ARRI. uta. edu/acs Lewis@uta. edu

Bruno Borovic MEMS Modeling and Control Thermal Model With Ai Qun Liu, NTU Singapore Mechanical Model Electrical Model FEA Optical Model Experiment

Micro Electro Mechanical Optical Systems - MEMS Electrostatic Comb Drive Actuator for Optical Switch Control Optical fibre Size of human hair 150 microns wide Electrostatic Parallel Plate Actuator Use feedback linearization to solve the pull in instability problem

Nonlinear Intelligent Control Objective: Develop new control algorithms for faster, more precise control of Do. D and industrial systems Confront complex systems with backlash, deadzone, flexible modes, vibration, unknown disturbances and dynamics Ø Ø Three patents on neural network learning control Internet- Remote Site Control and Monitoring Rigorous Proofs and Design Algorithms Applications to Tank Gun Barrels, Vehicle Active Suspension, Industrial machine control Ø SBIR contracts for industry implementation Ø Numerous Books and Papers Ø Award-Winning Ph. D students Nonlinear learning loops PID loop Intelligent Control Tools Neural Network Learning Controllers for Vibratory Systems- Gun barrel, HMMWV suspension

Relevance - Machine Feedback Control High-Speed Precision Motion Control with unmodeled dynamics, vibration suppression, disturbance rejection, friction compensation, deadzone/backlash control Industrial Machines Military Land Systems Vehicle Suspension Aerospace

The Perpetrators Aydin Yesildirek- Neural networks for control S. Jagannathan- DT NN control Sesh Commuri- CMAC, fuzzy Rafael Fierro- robotic nonholonomic systems. Hybrid systems Rastko Selmic- actuator nonlinearities Javier Campos- fuzzy control, actuator nonlinearities Ognjen Kuljaca- implementation of NN control

Neural Network Robot Controller Universal Approximation Property Feedback linearization. . qd Nonlinear Inner Loop Feedforward Loop qd ^ f(x) e [ I] r Kv q Robot System Robust Control v(t) Term PD Tracking Loop Problem- Nonlinear in the NN weights so that standard proof techniques do not work Easy to implement with a few more lines of code Learning feature allows for on-line updates to NN memory as dynamics change Handles unmodelled dynamics, disturbances, actuator problems such as friction NN universal basis property means no regression matrix is needed Nonlinear controller allows faster & more precise motion

Extension of Adaptive Control to nonlinear-in parameters systems No regression matrix needed Can also use simplified tuning- Hebbian Forward Prop term? Backprop terms. Werbos Extra robustifying terms. Narendra’s e-mod extended to NLIP systems

DT Systems- Jagannathan Add an extra feedback loop Two NN needed Use passivity to show stability Backstepping. . qd Nonlinear FB Linearization Loop NN#1 e. e= e qd = qd. qd [ I] r ^ (x) F 1 Kr Robust Control Term vi(t) 1/KB 1 id h Kh ue Robot System q qr =. r qr i ^ (x) F 2 NN#2 Tracking Loop Backstepping Loop Neural network backstepping controller for Flexible-Joint robot arm Advantages over traditional Backstepping- no regression functions needed DT backstepping- noncausal- Javier Campos patent

Dynamic inversion NN compensator for system with Backlash Estimate of Nonlinear y (n) Function d f$( x ) - & des [0 T ] Filter x e d - [ T I] v - 2 Backlash ˆj r K v v 1 K des ˆ u 1/s b - - y nn NN Compensator xd r ˆ Z F Backstepping loop U. S. patent- Selmic, Lewis, Calise, Mc. Farland Nonlinear System x

Force Control Flexible pointing systems SBIR Contracts Vehicle active suspension Andy Lowe Scott Ikenaga Javier Campos

ARRI Research Roadmap in Neural Networks 3. Approximate Dynamic Programming – 2006 Nearly Optimal Control Based on recursive equation for the optimal value Usually Known system dynamics (except Q learning) Extend adaptive control to The Goal – unknown dynamics yield OPTIMAL controllers. On-line tuning No canonical form needed. Optimal Adaptive Control 2. Neural Network Solution of Optimal Design Equations – 2002 -2006 Nearly Optimal Control Based on HJ Optimal Design Equations Known system dynamics Preliminary Off-line tuning Nearly optimal solution of controls design equations. No canonical form needed. 1. Neural Networks for Feedback Control – 1995 -2002 Extended adaptive control Based on FB Control Approach Unknown system dynamics to NLIP systems On-line tuning No regression matrix NN- FB lin. , sing. pert. , backstepping, force control, dynamic inversion, etc.

Murad Abu Khalaf System H-Infinity Control Using Neural Networks Performance output disturbance z Measured output d y u control where L 2 Gain Problem Find control u(t) so that For all L 2 disturbances And a prescribed gain g 2 Zero-Sum differential game

Cannot solve HJI !! Successive Solution- Algorithm 1: Let g be prescribed and fixed. Murad Abu Khalaf a stabilizing control with region of asymptotic stability 1. Outer loop- update control Initial disturbance 2. Inner loop- update disturbance Solve Value Equation Consistency equation For Value Function Inner loop update disturbance go to 2. Iterate i until convergence to with RAS Outer loop update control action Go to 1. Iterate j until convergence to , with RAS CT Policy Iteration for H-Infinity Control--- c. f. Howard

Results for this Algorithm The algorithm converges to the optimal solution on the RAS Sometimes the algorithm converges to the optimal HJI solution V*, For this to occur it is required that For every iteration on the disturbance di one has the value function increases the RAS decreases For every iteration on the control uj one has the value function decreases the RAS does not decrease , u*, d*

Murad Abu Khalaf Problem- Cannot solve the Value Equation! Neural Network Approximation for Computational Technique Neural Network to approximate V(i)(x) Value function gradient approximation is Substitute into Value Equation to get Therefore, one may solve for NN weights at iteration (i, j)

Murad Abu Khalaf Neural Network Optimal Feedback Controller Optimal Solution A NN feedback controller with nearly optimal weights

Finite Horizon Control Fixed-Final-Time HJB Optimal Control Optimal cost Optimal control This yields the time-varying Hamilton-Jacobi-Bellman (HJB) equation Cheng Tao

HJB Solution by NN Value Function Approximation Cheng Tao Time-varying weights Note that where Approximating is the Jacobian in the HJB equation gives an ODE in the NN weights Solve by least-squares – simply integrate backwards to find NN weights Control is

H-infinity OPFB control Theorem Necessary and Sufficient Conditions for Bounded L 2 Gain OPFB Control: Jyotirmay Gadewadikar V. Kucera, Lihua Xie

H-Infinity Static Output-Feedback Control for Rotorcraft J. Gadewadikar*, F. L. Lewis*, K. Subbarao$, K. Peng+, B. Chen+, *Automation & Robotics Research Institute (ARRI), University of Texas at Arlington +Department of Electrical and Computer Engineering, National University of Singapore

Discrete-Time Optimal Control Adaptive Dynamic Programming cost Value function recursion = the prescribed control input function Hamiltonian Optimal cost Bellman’s Principle Optimal Control System dynamics does not appear Solutions by Comp. Intelligence Community

Asma Al-Tamimi ADP for H∞ Optimal Control Systems Penalty output z Measured output y Disturbance d u Control where Find control u(t) so that for all L 2 disturbances and a prescribed gain g 2 when the system is at rest, x 0=0.

Discrete-Time Game Dynamic Programming: Backward-in-time Formulation • Consider the following continuous-state and action spaces discrete -time dynamical system • The zero-sum game problem can be formulated as follows: • The goal is to find the optimal strategies (State-feedback) for this multi-agent problem • Using Bellman optimality principle “Dynamic Programming”

Asma Al-Tamimi HDP- Linear System Case Value function update Solve by batch LS or RLS Control update Control gain A, B, E needed Disturbance gain Showed that this is equivalent to iteration on the Underlying Game Riccati equation Which is known to converge- Stoorvogel, Basar

Linear Quadratic case- V and Q are quadratic Asma Al-Tamimi Q learning for H-inf Q function update Control Action and Disturbance updates A, B, E NOT needed

Asma Al-Tamimi

Continuous-Time Optimal Control System Cost Hamiltonian Optimal cost Bellman Optimal control HJB equation c. f. DT value recursion, where f(), g() do not appear

CT Policy Iteration Utility Cost for any given u(t) Lyapunov equation Iterative solution Pick stabilizing initial control Find cost Update control • Convergence proved by Saridis 1979 if Lyapunov eq. solved exactly • Beard & Saridis used complicated Galerkin Integrals to solve Lyapunov eq. • Abu Khalaf & Lewis used NN to approx. V for nonlinear systems and proved convergence Full system dynamics must be known

LQR Policy iteration = Kleinman algorithm 1. For a given control policy solve for the cost: Lyapunov eq. 2. Improve policy: § If started with a stabilizing control policy the matrix monotonically converges to the unique positive definite solution of the Riccati equation. § Every iteration step will return a stabilizing controller. § The system has to be known. Kleinman 1968

Policy Iteration Solution Policy iteration This is in fact a Newton’s Method Then, Policy Iteration is Frechet Derivative

Draguna Vrabie Now Greedy ADP can be defined for CT Systems Solving for the cost – Our approach ADP Greedy iteration 1. For a given control policy update the cost: 2. Improve policy (discrete update of a continuous time controller): No initial stabilizing control needed § the cost is convergent to the optimal cost § the controller is stabilizing the plant after a number of iterations u(t+T) in terms of x(t+T) - OK § any initial §

Draguna Vrabie Analysis of the algorithm For a given control policy Greedy update with is equivalent to a strange pseudo-discretized RE Can show When ADP converges, the resulting P satisfies the Continuous-Time ARE !! ADP solves the CT ARE without knowledge of the system dynamics f(x)

Draguna Vrabie Analysis of the algorithm Lemma 2. CT HDP is equivalent to This extra term means the initial Control action need not be stabilizing Equivalent to underlying equation a strange pseudo-discretized RE

Solve the Riccati Equation WITHOUT knowing the plant dynamics Model-free ADP Direct OPTIMAL ADAPTIVE CONTROL Works for Nonlinear Systems Proofs? Robustness? Comparison with adaptive control methods?

DT ADP vs. Receding Horizon Optimal Control Forward-in-time ADP Backward-in-time optimization – 1 -step RHC

New Books Recent Patent J. Campos and F. L. Lewis, "Method for Backlash Compensation Using Discrete-Time Neural Networks, " SN 60/237, 580, The Univ. Texas at Arlington, awarded March 2006. Use filter to overcome DT backstepping noncausality problem

International Collaboration Jie Huang Sam Ge

Organized and invited by Professor Jie Huang, CUHK SCUT / CUHK Lectures on Advances in Control March 2005

Sponsored by IEEE Singapore SMC, R&A, and Control Chapters Organized and invited by Professor Sam Ge, NUS UTA / IEEE Singapore Short Course Wireless Sensor Networks for Monitoring Machinery, Human Biofunctions, and Biochemical Agents F. L. Lewis, Assoc. Director for Research Moncrief-O’Donnell Endowed Chair Head, Controls, Sensors, MEMS Group Automation & Robotics Research Institute (ARRI) The University of Texas at Arlington

Mediterranean Control Association Founded 1992 Founding Members M. Christodoulou P. Ioannou K. Valavanis F. L. Lewis P. Antsaklis Ted Djaferis P. Groumpos Conferences held in Crete, Cyprus, Rhodes Israel Italy Dubrovnik, Croatia Kusadasi Turkey 2004 Bringing the Mediterranean Together for Collaboration and Research