SIMULTANEOUS PERTURBATION STOCHASTIC APPROXIMATION FOR REAL-TIME OPTIMIZATION OF MODEL PREDICTIVE CONTROL
Irina Baltcheva
Felisa J.V´a zquez-Abad(member of GERAD)
Universit´e de Montr´e al
DIRO,CP6128succ Centre Ville,
Montreal,QC,H3C3J7Canada
baltchei,vazquez@iro.umontreal.ca
Smaranda Cristea
C´e sar De Prada
Universidad de Valladolid DISA,P Prado de la Magdalena s/n, Valladolid,47005Spain smaranda,prada@autom.uva.es
KEYWORDS
SPSA,Non-linear Model Predictive Control. ABSTRACT
The aim of this paper is to suggest a global optimization method applied to a model predictive control problem.We are interested in the Van der Vusse reaction found in multiple chemical processes.The control variables(temperatures and inputflow rates)are real time continuous processes and a tar-get reference level must be reached within certain operational constraints.The canonical model discretises time using a sampling interval,thus translating the control problem into a non-linear optimization problem under constraints.Because of the non-linearity of the cost function,common methods for constrained optimization have been observed to fail in experiments using ECOSIM to simulate the production pro-cess.The controllers do not achieve their optimal values and the numerical optimization based on approximating gradi-ents and hessians cannot be performed in real time for the op-eration of the plant to be successful.In this research we im-plement a methodology for global optimization adding noise to the observations of the gradients,which will perform much better than deterministic methods.
INTRODUCTION
Model Predictive Control(MPC)is now recognized in the industrial world as a proven technology,capabl
e of dealing with a wide range of multivariable constrained control prob-lems.Nevertheless,most of the industrial controllers are based on linear internal models which limits its applicability. Because of it,non-linear model predictive control(NMPC) has received a lot of attention in the latest years,both from the point of view of its properties[2]and implementation. Referring to this last aspect,the main drawback is the com-putational burden that NMPC implies.While linear MPC with constraints can solve the associated optimization prob-lem each sampling time using QP or LP algorithms for which very efficient codes are available,NMPC relies on non-linear programming(NLP)methods such as SQP,that are known to be far more CPU demanding.Several schemes have been proposed to deal with this problem,among them the well known sequential and simultaneous approaches.1 For sequential solutions,the model is solved by integra-tion at each iteration of the optimization routine.Only the control parameters remain as degrees of freedom in the NLP. Simulation and optimization calculations are performed se-quentially,one after the other.The approach can easily be coupled with advanced simulation tools.In contrast,simul-taneous model solution and optimization includes both the model states and controls as decision variables and the model equations are appended to the optimization problem as equal-ity constraints.This can greatly increase the size of the op-timization problem,leading to a trade-off between the two approaches.In both cases,computation time remains a diffi-culty in order to implement NMPC in real processes.
This paper shows a global optimization method oriented to reduce the difficulties associated with the computation of the gradients,in order to facilitate the implementation of NMPC algorithm,using the sequential approach,applied to a benchmark problem:Van der Vusse reactor.
MODEL DESCRIPTION
The Van der Vusse reaction is described in detail in[4]and the references therein.To summarize,there is a substance in input,a chemical reaction and a substance in output.
We’ll denote by:
the concentration of product(controlled)
Figure1:Van der Vusse
the concentration of product(measured)
the temperature in the reactor(measured)
the temperature in the coolant(measured)
the inputflow of product(manipulated)
the heat removal(manipulated)
constants.
To the input we associate the control variables and. The controlled variable is associated with the output. Thereby,we want to control the output by manipulating the input and.
To describe the evolution of this dynamical system,we in-troduce the non-linear differential equations related to mass and energy conservation:
The concentration of product must not exceed some upper and lower limits.This gives the following constraints,which will be included in the objective function later on: MODEL BASED PREDICTIVE CONTROL
Let us denote by:
reactor thenthe state at time
the controlled variable at time
the control at time(manipulated vari-able)
t
r
y(t)
T
Figure2:Control
:reference level.
The objective of the predictive control is tofind the future optimal control sequence
over afinite horizon time,which minimizes the quadratic error between the controlled variable and its reference level.In addition,must assure that the trajectory of
is he range of values taken by the manipulated variables and varies gradually and does not go from very high values to very low ones.The objective function is then:
where
r
2h y(t)
h t
N_u
Figure3:Discretized Control
applied to the manipulated and controlled variables,can be written as:
After penalizing the constraints on the controlled variable, the objective function becomes:
where and are non-negative constants,and,repre-sent a penalty function.Remark that this optimization prob-lem cannot be solved analytically.In fact,it is hard to calcu-late the gradient,because is non-linear i
n.This is why we need to estimate it.
In order to treat the control constraints,we’ll make a pro-jection of the Lagrangian function over the set of feasible controls.We’ll truncate it in the sense that if, we’ll set,and if,we’ll set .
Previous approaches to the optimization problem were based on the SQP algorithm implemented in the NAG li-brary,which usesfinite differences to estimate the gradient .This approach has the disadvantage of loosing preci-sion and is quite slow because of the large number of func-tion evaluations.Also,it may get trapped in local minima. SYSTEM REQUIREMENTS
To see how fast the optimization must be,let us summa-rize the simulation by the algorithm below:
WHILE(simulation time)DO
WHEN(sampling)DO
1.Calculate the optimal control(
);
2.Apply the control;
3.Measure;;
4.Take new value of;;
5.;go to1.
END WHEN
END WHILE
It is clear that the optimal control must be found in less than seconds(the sampling time)if we want the control to be done online.The optimization must be fast and precise.
The Van der Vusse model presents important non-linearities which makes the problem quite difficult.Another source of problems is the possible existence of local minima. Among others,this is a reason why we’ll apply a method of global optimization.In the next section,we present this method,which is known for its efficiency in high dimen-sional problems.It could replace successfully thefinite dif-ferences approximations in the Van der Vusse model and will perform as well in the presence of a larger number of vari-ables.
SIMULTANEOUS PERTURBATION STOCHASTIC APPROXIMATION(SPSA)
SPSA is a descent method capable offinding global min-ima.Its main feature is the gradient approximation that re-quires only two measurements of the objective function,re-gardless of the dimension of the optimization problem.Re-call that we want tofind the optimal control,with loss function:
Both Finite Differences Stochastic Approximation(FDSA) and SPSA use the same iterative process:
where represents the iterate,is the estimate of the gradient of the objective function
vector,the component of the symmetric finite difference gradient estimator is:
FD:
Remark that FD perturbs only one direction at the time,while
the SP estimator disturbs all directions in the same time (the numerator is identical in all components).The number of loss function measurements needed in the SPSA method for each is always 2,independent of the dimension .Thus,SPSA uses times fewer function evaluations than FDSA,which makes it a lot more efficient.
-10-5
5
10
-10
-5
05
10
u _2
u_1
’J(u).dat’
g1(x)
g2(x)
’’
’’
Figure 4:SPSA vs FDSA
Simple experiments with showed that SPSA con-verges in the same number of iterations as FDSA.The latter follows approximately the steepest descent direction,behav-ing like the gradient method (see Figure 4).On the other hand,SPSA,with the random search direction,does not fol-low exactly the gradient path.In average though,it tracks it nearly because the gradient approximation is an almost un-biased estimator of the gradient,as shown in the following lemma found in [3].
Lemma 1Denote by
the bias in the estimator .Assume that are
all mutually independent with zero-mean,bounded second moments,and uniformly bounded on .Then
w.p.1.
The detailed proof is in [3].The main idea is to use condi-tioning on to express and then to use a second
order Taylor expansion of and .After algebraic manipulations implying the zero mean and the independence of ,we get
The result follows from the hypothesis that .
Next we resume some of the the hypotheses under which converges in probability to the set of global minima of .For details,see [5],[3]and [6].The efficiency of the method depends on the shape of ,the values of the parameters and and the distribution of the perturbation terms .First,the algorithm parameters must satisfy the following conditions:
-
,when
and
;a
good choice would be
Figure5:Concentration of Product B
two times faster when using SPSA.Like expected,the latter
needed times less cost function evaluations(in fact,
the dimension of the problem here is2:).The resulting control can be seen in Figure5,where are plotted
the reference level changing over time,the upper and lower
bounds of the controlled variable and the concentration of product B()itself.The small perturbation at time0.7 is due to the change of temperature provoked on purpose. We see that the control is handling it well.Though,the con-trolled variable seems having some difficulties when near its upper and lower bounds,indicating that our way of treating the limit constraints may be incorrect.We used a dynamic stopping criteria of the form,where in most of the experiments,implying a small number of SPSA iterations.A higher precision was very costly,indicating that what we gained in speed was unfortunately lost in precision. However,we believe that a better choice of parameters can make the method more robust and this makes part of our on-going work.
CONCLUSION
In this research we present a methodology for global op-timization adding noise to the observations of the gradients in order to achieve better performance of the model.Theory showed that the addition of random noise can make the con-trol variables attain near optimality much faster than deter-ministic methods as confirmed by some of our experiments. In addition,this method can provably overcome the curse of dimensionality and thus be used in larger problems.How-ever,finding the suited parameters proved to be challenging, showing that a particular attention should be given to this point.References
[1]  D.P.Bertsekas.Nonlinear Programming.Mass.USA,athena
scientific edition,1999.
[2]H.Chen and F.Allgower.A quasi-infinite horizon nonlinear
model predictive control scheme with guaranteed stability.In Automatica,1998.
[3]M.C.Fu and S.D.Hill.Optimization of discrete event systems
via simultaneous perturbation stochastic approximation.IEEE Transactions,29:233–243,1997.
[4]  A.K.H.Chen and F.Allgower.Nonlinear predictive control
of a benchmark cstr.In Proceedings of3rd European Control Conference,pages3247–3252,1995.
[5]J.L.Maryak and D.C.Chin.Global random optimization by
simultaneous perturbation stochastic approximation.In Pro-ceedings of the American Control Conference,pages756–762, 2001.
[6]J.C.Spall.An overview of the simultaneous perturbation
method for efficient optimization.John Hopkins APL Tech-nical Digest,19(4):482–492,1998.
[7]  C.D.P.W.Colmenares,S.Cristea and al.Mld systems:Mod-
eling and control experience with a pilot process.In Proceed-ings of IMECE2001,New York,USA,2001. ACKNOWLEDGEMENTS
The work of thefirst two authors was sponsored in part by NSERC and FCAR Grants of the Government of Canada and Quebec.

版权声明：本站内容均来自互联网，仅供演示用，请勿用于商业和其他非法用途。如果侵犯了您的权益请与我们联系QQ:729038198，我们将在24小时内删除。