Optimal control of nonlinear systems with constraints.

Jadlovska, A. ; Katalinic, B. ; Hrubina, K. 等

1. Introduction

Methods development of an automatic control of systems is connected with the development of optimizing methods. optimizing methods enable the development of such control methods as an adaptive control method, robust control method, predictive control method, and intelligent control method. The development of optimizing methods itself is based on iterative gradient methods, conjugated gradients as well as on a changeable metric. Algorithms designed based on the mentioned optimizing methods enable to quicken the processes of searching for an extreme of a criterion function in a minimum number of steps. Algorithms of optimizing methods which make use of the principles of projection enable to solve optimization problems under the fulfillment of limiting conditions which evidently has a great influence on the development of the whole group of modern control methods that for the calculation of the control effect utilize a mathematical model of the process determined based on the measured process data (MPC methods and algorithms). At present, a great success has been achieved by agent and colonic control methods as well as methods and genetic algorithms used in the system of off-line techniques applied to the searching for a global extreme of a criterion function. The applications of the mentioned methods and their algorithms to nonlinear processes modeling and control, systems, significantly contribute to the improvement of approximation properties of modeling processes as well as to the improvement of a control quality of selected types of a control (Athans & Falb, 1986), (Hrubina & Jadlovska, 2002), (Hrubina et al., 2010), (Jadlovska & Hrubina, 2011). The aim of the contribution is, based on the defined problems of a control of nonlinear systems with limitations, to present the use of iterative optimizing methods. We also aimed at the solution of the Lagrange problem through the method that converts the problem to the optimization problem of the theory of games. In order to find a saddle point of a functional, an algorithm of an iterative method is used. Another aim is to present the algorithms designed based on gradient methods applying projection principles to the solution of mathematical mode the tasks of processes optimal control.

2. Formulation of problems to solve

Let us suppose that the dynamics of a nonlinear system of processes is expressed by a system of nonlinear differential equations of the form

[??](t|u)= f (x(t|u), u(t), t) (1)

with the initial condition x(0|u) = [x.sub.0] where, x = ([x.sub.1], [x.sub.2], ..., [x.sub.n]) is a state vector of a controlled system u = ([u.sub.1], [u.sub.2], ..., [u.sub.r]) is a control vector, t is time, f(., .) is n-dimensional vector function, which is continuously differentiable; further on, there is given a set of constraining conditions

x [member of] X [subset] [R.sub.n]

u [member of] U [subset] [R.sub.r]; t [member of] <[t.sub.0], T> (2)

where X is a closed domain in n-dimensional Euclidean space [R.sub.n] (state space). U a closed domain in r-dimensional Euclidean space [R.sub.r] (space of a control), [t.sub.0] the beginning, T the moment of a controlled process completion; and finally, a functional is defined in the form

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (3)

in which the functions g(x, u, t), [phi](x) are scalar functions. The task is to find such a control, which would transfer a controlled system from any initial state x([t.sub.0]) = [x.sub.0] [member of] X to the environment of the final state x(T) = [x.sub.T] [member of] X, it satisfies the set of constraining conditions (2) and minimizes a functional (3), which is called a criterion for optimal control. This problem in which a functional is defined by a criterion of the form (4) is called the Bolza problem. In the role of a terminal control, instead of a functional (3), there is a functional of the form

[J.sub.M](u) = [phi](x(T|u)) (4)

The problem defined by a system of nonlinear differential equations (1) limiting conditions(2) and a functional (4) is called the Mayer problem. In case that the functional (3) is expressed in the form

[J.sub.L](u) = [[integral].sup.T.sub.0] g(x(t|u), u(t), t)dt (5)

where g(x, u, t) is a scalar function, continuously differentiated according to x and u. In the domain [R.sub.n] x [R.sub.r], the problems (1), (2) with a functional (5) are called the Lagrange problem. Based on the theory we know that the Bolza and Mayer optimization problems can be converted to the lagrange optimization problem, where subintegral functions in criteria [J.sub.M](u) and [J.sub.B](u) will contain a gradient of the function [phi](x). In practical problems, there are often limitations to a control u(t), of which let us state the following expressions

1) [absolute value of [u.sub.j](t)] [less than or equal to] [[rho].sub.j](t) j = 1, ..., r

2) [r.summattion over (j = 1)][[alpha].sub.j](t)[u.sup.2.sub.j](t) [less than or equal to] [[rho].sup.2](t)

3) [u(t), N(t)u(t)] [less than or equal to] [[rho].sup.2](t)

4) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

5) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

where a scalar product is denoted as [., .], [[rho].sub.j](t) [greater than or equal to] 0, [[alpha].sub.j](t) > 0, [rho](t) and N(t) is a square matrix of the r-th order, [[rho].sub.j] and [rho] given positive constants, (Bellman, 1967), (Boudarel & Delmas, 1987), (Golder & Kubik, 1983), (Huba, 1999), (Cea, 1971).

3. Optimality conditions

For the Lagrange problem, the optimality conditions can be expressed by a theorem.

Theorem 1 If a control u(t) [member of] U is supposed to be a solution to the Lagrange problem, it is necessary that the following conditions are satisfied

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (6)

It is obvious that the condition (6) will be valid for the Bolza and Mayer problems considering that the function G will be substituted for the function [G.sub.B] or [G.sub.M]. Similarly, we can state the optimality condition also with the use of a maximum principle.

Theorem 2 An optimal control u(t) in the Lagrange problem, optimal trajectory x(t|u)and optimal vector function [psi](t|u) for t [member of] <0, T> are the solutions to (2n + r) systems of equations

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (7)

where the function F(x, [psi], u, t) is the right side of the differential equation [psi](t|u) where [psi](t|u) satisfies the condition of a transcendent [psi](T|u) = 0 thus

F(x(t|u), [psi](t|u), u(t), t) = = - ([[nabla]sub.x]f(x(t|u), u(t)), t)) * [psi](t|u) + [[nabla]sub.x]g(x(t|u), u(t), t) (8)

4. Iterative gradient methods of nonlinear systems optimization

In order to find an optimal control u(t), an optimal trajectory x(t|x) and a vector function [psi](t|u) in the Lagrange problem for t [member of] <0, T>, the relations (7) i.e. (2n + r) equations were presented in the previous paragraph. Based on iterative gradient methods we will calculate the sequence of a control [u.sup.1](t), [u.sup.2](t), ..., [u.sub.k](t), [u.sup.k+1] (t), based on the recurrent relations

[u.sup.k+1](t) = [u.sup.k] (t) + [[lambda].sub.k] [[[bar.u].sup.k](t) - [u.sub.k](t)] (9)

and the corresponding trajectories x(t|[u.sup.1]), x(t|[u.sup.2]), ..., x(t|[u.sup.k]), x(t|[u.sup.k+1]), ..., and the auxiliary vector functions [psi](t|[u.sup.1]), [psi](t|[u.sup.2]), ..., [psi](t|[u.sup.k]), where x(t|[u.sup.k]) is the solution to the system of integral equations (the first equation (7)), [psi](t|[u.sup.k]) is the solution to the system of integral equations (the second equation (7)) and finally [[bar.u].sup.k](t) is the solution to the last equation in the relation (7). It is known that the value of the step [[lambda].sub.k] can be selected in a different way (Hrubina, 2001), (Hrubina & Jadlovska, 2002).

5. Optimizing methods applied to the solution of problems of control with limitations

In optimization problems of an automatic control system with a fixed time for a process control, besides the limitations to a control u(t) [member of] U (for example, the five mentioned types) there can also be the limitations of the form

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (10)

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (11)

[f.sub.i](x(t|u), u(t), t)= 0, i = 1, ..., [n.sub.1], (12)

[f.sub.i](x(t|u), u(t), t) [less than or equal to] 0, i = [n.sub.1] + 1, ..., [n.sub.1] + [n.sub.2] (13)

where the functions [g.sub.i](x, u, t) and f(x, u, t) are within the interval t [member of] <0, T> continuous and continuously differentiable according to x and u in [R.sub.n] x [R.sub.r] and the natural numbers [m.sub.1], [m.sub.2], [n.sub.1] a [n.sub.2] [greater than or equal to] 0 are finite. In case that the limitations (10) to (13) occur in the Lagrange problem for the minimization of the functional [J.sub.L] (u) of the form (5), in order to solve a mathematical model of the process (1), (2) with a control u(t) [member of] U the two methods are used

1) the method which converts the Lagrange problem to an optimization problem of the theory of games.

2) The method of embedding, which resides in the fact that the Lagrange problem with the limitations (10) to (13) is converted to the problem of searching for the control [u.sup.[omicron].sub.[psi]](t) [member of] U based on the condition of a functional minimization [J.sub.[psi]](u) solving the systems (1) and (2):

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (14)

The essence of the method of the Lagrange multiplications is that the Lagrange problem with the limitations (10) to (13) is substituted for the problem of finding a saddle point u(t), [psi](t) of the functional

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (15)

satisfying the conditions

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (16)

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

Theorem 3 Let ([u.sup.[omicron]], [[psi].sup.[omicron]] [member of] U x [PSI] be such a couple of elements for which the equations (6) are satisfied and let the functional J (u, [psi]) be convex-concave owing to u and [psi]. Then ([u.sup.[omicron]], [y.sup.[omicron]]) is the solution to the defined problem, i.e. ([u.sup.[omicron]], [y.sup.[omicron]]) is a saddle point of the functional J (u, [psi]), (Jadlovska & Hrubina, 2005), (Macura & Macurova, 2009).

The functional F([u.sup.m], [[psi].sup.m]) satisfies Theorem 3, it has a derivation up to the second order and also satisfies the Lipschitz condition. In order to calculate the sequence of the couples ([u.sup.1], [[psi].sup.1]), ([u.sup.2], [[psi].sup.2]), ..., ([u.sup.m+1], [[psi].sup.m+1]), the following recurrent relations will be used in an algorithm

[u.sup.m+1] = [u.sup.m] - [[lambda].sub.k]f([u.sup.m], [[psi].sup.m]) (17)

[[psi].sup.m + 1] = [[psi].sup.m] - [[lambda].sub.k] g([u.sup.m], [[psi].sub.m])

in which the value of the step [[lambda].sub.k] > 0 is selected in various ways based on certain conditions.

6. Iterative algorithm of a gradient method used to determine a saddle point of a functional with constraints

Step 1 Selection of initial values [u.sup.1], [[psi].sup.1] and a value of a parameter of the numerical method [epsilon].

Step 2 Initiation of a control variable of the cycle m = 1.

Step 3. Calculation J([u.sup.m], [[psi].sup.m]), [g.sub.1]([u.sup.m], [[psi].sup.m]), [g.sub.2]([u.sup.m], [[psi].sup.m]).

Step 4 If the condition F([u.sup.m], [[psi].sup.m]) < [epsilon] is satisfied proceed to Step 8, otherwise continue to Step 5.

Step 5 Selection of the step [[lambda].sub.k] value.

Step 6 Calculation of the values of the elements [u.sup.m + 1], [[psi].sup.m + 1] according to the recurrent relations (17).

Step 7 Write m:= m +1 continue to Step 3.

Step 8 u = [u.sup.m], [psi] = [[psi].sup.m].

Step 9 End of the calculation.

Algorithm of an iterative gradient method can also be used to search for a functional saddle point J(u, [psi]) with a limitation (u, [psi]) [member of] [PSI]. In such a case, when satisfying the condition in the 4-th Step, we verify the fulfillment of another condition ([u.sup.m], [[psi].sup.m]) [member of] [PSI] and in case that it is not satisfied, we define the functions based on which we calculate [u.sup.m + 1], [[psi].sup.m + 1] and then we proceed to Step 7 otherwise we continue to Step 8, (Hrubina&Jadlovska, 2002),(Jadlovska, 2003)

7. Mathematical model of the tasks of static and dynamic optimization

7.1 Formulation of the static optimization problem

Let us have the set D [subset] [R.sub.n] and the function of multivariables [X.sub.1], [X.sub.2], ..., [x.sub.n] which are indicated

J([x.sub.1], [x.sub.2], ..., [x.sub.n]) [member of] R, or J(x) [member of] R, x [member of] D.

Then we search for [x.sub.0] so that the following is satisfied

[x.sub.0] [member of] D, J([x.sub.0]) [less than or equal to] J(x), [for all]x [member of] D [subset] R (18)

where D is the set representing possible solutions to achieve the determined goal.

The solutions that belong to the set D are considered to respect certain constrains and the set D is often called the domain which is created by the constrains. Let the domain D be defined by the following constraints

a) a system of equations

[g.sub.i](x) = 0, pre i = 1, 2, ..., m (19)

where x is a column vector, x = [([x.sub.1], [x.sub.2], ..., [x.sub.n]).sup.T] [member of] D [subset] [R.sub.n].

b) a system of inequalities

[h.sub.j](x) [less than or equal to] 0, for j = 1, 2, ..., m (20)

The function J(x) represents the criterion (a cost function), which provides information about the possible solution. The formulation of the optimization problem (18), (19) or (18), (20) addresses investigation of the solution [x.sub.0], in which the cost function J(x) obtains optimum value. Optimization problems are classified according to the criterion function (linear, non-linear); and constraints g(x), h(x). In general, we speak about the problems of programming (see the table)

Note: The literature often presents the optimization problems under the heading linear programming problems or non-linear programming problems (under the ordinals 3, 4).

Since the variables within the defined optimization problems the criterion function as well as within the constraints are not dependent on the time, such optimization problems are also called the problems of static optimization. They also include the where the variables are independent on the time. In order to solve a specific problem related to the static optimization, the method is chosen according to the type of the problem. A simplex algorithm is used to solve the problem of a linear programming. If a specific problem is related to a non-linear programming, then the dimensionality of the problem is decisive when choosing the method. With one-dimensional optimization problems the following methods can be used

* investigative methods,

* deterministic investigation with a firm or an adaptive strategy,

* dichotomous investigation,

* the Fibonacci numbers method.

In order to solve the problems of a dynamic optimization, in which the variables are dependent on time, the methods or algorithms based on the calculus of variations, the maximum principle as well as the

Bellman principle are used.

the statistical methods are applied to solve optimization problems of a statistical character

* analysis of regression,

* correlation analysis,

* scheduled experiment, (Hrubina, 2001), (Jadlovska, 2000), (Jadlovska, 2003) (Jadlovska, 2005).

7.2 Formulation of the dynamic optimization problem

The principal task of the dynamic optimization is to determine the vector u(t) of the control variables of the real system in dependence on time t so that the functional

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (21)

obtains the extreme value and the following conditions are satisfied

d[x.sub.i](t)/dt = [f.sub.i](x(t)), i = 1, 2, ..., n (22)

x([t.sub.0]) = [x.sub.0], x([t.sub.1]) = [x.sub.1]

and [g.sub.j](x(t), u(t)) [less than or equal to] 0, j = 1, 2, ..., m (23)

where x(t) is a state vector and u(t) is a vector of a real system control. The vectors x(t), u(t) are apparently dependent on time, [f.sub.i](x(t), u(t)) and [g.sub.j] (x(t), u(t)) are the variables functions u(t) and x(t) non-linear in general. components of the vectors x(t), u(t) belong to the convex sets and f(x(t), u(t)) is a convex function for each x(t) and u(t). functional J(x(t), u(t)) (21) is the criterion of a dynamic optimization. Differential equations (22) express a dynamic model of the investigated process, while the inequalities (23) represent the constraints of the control area, (ekeland & Teman, 1976), (Hrubina, 2001).

8. Algorithms of gradient methods

In the following part of the paper we suppose that the reader has theoretical knowledge of iterative methods presented in ref., (Athans&Falb, 1976), (Boudarel et al.,1977). We will deal with the solution of optimization problems in which the mathematical model of the process, object is expressed by:

* a cost function J(x), (18),

* a cost function J(x) under the conditions of the equality type (19),

* a cost function J(x) under the conditions of the inequality (20).

8.1 The essence of the gradient method algorithm

Let us search for the minimum of the convex cost function J(x). Let us have initial the initial point [x.sup.(0)], the coordinates of the next point are determined by means by means of the recurrence formula

[x.sup.(r + 1)] = [x.sup.(r)] + h[v.sup.(r)] (24)

where h is a chosen positive constant and v(r) is a chosen direction creating an acute angle with the direction of the gradient J(x). Let us indicate gradient of the cost function

[nabla]J(x) = grad J(x) = [J.sub.x](x) = [([partial derivative]J/[partial derivative][x.sub.1], ..., [partial derivative]J/[partial derivative][x.sub.n]).sup.T] (25)

We will consider a discrete gradient method since the optimization problems will be solved with PC. We obtain the gain of the cost function [DELTA]J(x), after the development J(x) to the series of Taylor is provided. The principle of the gradient methods lies in the creation of the so-called minimizing sequence {[x.sup.(r)]} of the recurrence relation

[x.sup.(r + 1)] = [x.sup.(r)] - h [nabla] J([x.sup.(r)]) (26)

where the constant h is chosen so that it is valid J([x.sup.(r + 1)]) < J([x.sup.(r)]) < ... < J([x.sup.(1)]) < J([x.sup.(0)]) In order to complete the calculation in the process of investigation of the minimum value of the cost function, the following condition can be used [absolute value of (J([x.sup.(r + 1)]) - J([x.sup.(r)]))] < [epsilon] where [epsilon] is a beforehand chosen small positive number. After the required accuracy of the point [x.sup.*]. is obtained, the iterative process of the calculation is stopped. We know that the investigated minimum [x.sup.*] according to the conditions of the theorem, must satisfy [nabla]J([x.sup.*]) = 0. And the matrix H([x.sup.*]) Hessian must have positive definite values.

8.2 Algorithm of the projective gradient method (conditions of the quality type)

In many branchy of science it is necessary to calculate the optimization problem of the following type.

Let us find the minimum of the cost function J(x), i.e.

J([x.sup.*]) = min J([x.sub.1], ..., [x.sub.n]), x [member of] D, (27)

Under the conditions of the equality type

[g.sub.i](x) = 0, i = 1, ..., p ; x [member of] D [subset] [R.sub.n], p < n. (28)

We suppose that the function J(x) and the function [g.sub.i](x) are in a given domain continuous and differentiable. Each point x, for which the conditions (28) are independent linear conditions will of which it follows that the matrix G having the elements ([g.sub.ij]) = [partial derivative][g.sub.i](x)/[partial derivative][x.sub.j] will be of the p-th order. In order to solve (27), (28), our aim is to create the iterative algorithm by the use of the cost function gradient in the point [x.sup.(r)] as well as the gradient of each function [g.sub.i](x) in the point [x.sup.(r)], (the coordinates of the point [x.sup.(r)] are either known or they are chosen for r = 0, i.e. for the initial point). Therefore we provide the development of the cost function (27) as well as that of the functions occurring in the conditions (28) to the series of Taylor

J(x + [delta]x) = J(x) + [nabla] [J.sup.T] (x) [delta]x + O([parallel] x [[parallel].sup.2])

g(x + [delta]x) = g(x) + G(x) [delta]x + O([parallel] x [[parallel].sup.2])

thus, the problem of minimizing (27), under the fulfillment of the conditions (28) is transformed to the problem of investigation such [delta] that

[[delta].sup.T][delta] = [d.sup.2], G[delta] = 0, [nabla]J(x)[delta] [right arrow] min (29)

presents the theoretical proof of the calculation procedure of (27), (28); in the following chapter we present the created algorithm of the projective gradient method with the instructions on the application to the solution of optimization problems (Hrubina, 2001). Let us create Lagrangian function [phi]([delta], [lambda], u) based on the equation (29) to express the vector [delta] and [lambda] in order to calculate the coordinates of the point [x.sup.(r + 1]).sub.[alpha]] in the iterative process according to the relation

[x.sup.(r + 1).sub.[alpha]] = [x.sup.(r)] - h {I - [G.sup.T]([x.sup.(r)]) (G([x.sup.(r)]) . [[G.sup.T]([x.sup.(r)])).sup.-1] . G([x.sup.(r)])} [nabla]J([x.sup.(r)]) (30)

To calculate [x.sup.(r + 1)] we use a correction step [[delta].sup.(r).sub.[alpha]], thus

[x.sup.(r+1)] = [x.sup.(r + 1).sub.[alpha]] + [[delta].sup(r).sub.[alpha]] (31)

hence we obtain

[x.sup.(r + 1)] = [x.sup.(r + 1).sup.[alpha]] - [G.sup.T]([x.sup.(r + 1).sub.[alpha]])(G([x.sup.(r + 1).sub.[alpha]]) [[G.sup.T]([x.sup.(r + 1).sub.[alpha]])).sup.-1] . [[gamma].sub.i] (32)

where [[gamma].sub.i] = [g.sub.i] ([x.sup.(r + 1).sub.[alpha]]).

In case that the calculated coordinates of the point, [x.sup.(r + 1)] owing to the relation (32) do not satisfy the conditions (28) with the satisfactorily accuracy, we perform another correction step and the same procedure is applied until the required accuracy is obtained. With the application of the gradient method algorithm, in each iterative step for the cost function and the functions [g.sub.i](x) we must calculate

* the gradient of the function J(x) in the point [x.sup.(r)], i.e. [nabla] J([x.sup.(r)])

* the gradient of each function [g.sub.i](x) in the point [x.sup.(r)], i.e. G([x.sup.(r)])

* the gradient of each function [g.sub.i](x) in the point [x.sup.(r + 1).sup.[alpha]], i.e. (G([x.sup.(r + 1).sub.[alpha]]);

According to the theoretical results and ref. (Jadlovska, 2003), (Jadlovska, 2005), (Hrubina, 2005) we will create algorithm of the projective gradient method

Step 1 Load the parameters values [epsilon], h and the values of the coordinates of the initial point [x.sup.(0)] = ([x.sup.(0).sub.1]), [x.sup.(0).sub.2], ..., [x.sup.(0).sub.n]).

Step 2 Calculate the value of the cost function J([x.sup.(0)]), its gradient [nabla]J([x.sup.(0)]) as well as calculate the individual elements of the matrix ([g.sub.ij]), i.e. the functions derivations [g.sub.i](x) in the point [x.sup.(0)], i.e. determine the matrix for the conditions. G([x.sup.(0)]).

Step 3 Calculate the projection of the direction vector v according to the relations A = G . [G.sup.T]; B = [A.sup.-1]

C = [G.sup.T]B; Q = C . G, t. j. v = (1 - Q) [nabla] J

Step 4. Calculate the coordinates of the new solution vector [x.sup.(r + 1).sub.[alpha]] = [x.sup.(r)] - hv; r = 0, 1,

Step 5 Calculate the coordinates of the deviation vector--gamma of the conditions, i.e. [[gamma].sub.i] = [g.sub.i]([x.sup.(r + 1).sub.[alpha]]); and the Euclidean norm of the gamma vector [parallel] [gamma] [parallel].

Step 6 In the case that the norm of the gamma vector satisfies the conditions, continue calculating, otherwise go on calculating the correction step: r = - C[gamma] and [x.sup.(r + 1)] according to (32) and continue with the Step 5.

Step 7 Calculate the difference between the values of the cost function [DELTA]J = J([x.sup.(r + 1)]) - J([x.sup.(r)]).

Step 8 In the case that [DELTA]J < 0 is satisfied, continue calculating, otherwise reduce the value of the step, h (e.g. in 1/2) and continue with the Step4.

Step 9 Increase the value of the step h (e.g. in 3/2 h).

Step 10 If the condition of the iterations completion is satisfied [absolute value of (J(x([x.sup.(r + 1)]) - J ([x.sup.(r)]))] < [epsilon]. Complete the process of calculation, otherwise save the values of the vector [x.sup.(r + 1)] in the memory [x.sup.(r)] and J([x.sup.(r + 1)]) in the memory J([x.sup.(r)]) and continue calculating Step 2.

8.3 Algorithm of the generalized method of reduced gradient (GMRG)

Let us consider the problem defined by the cost function (27) and the conditions of the equations type (28), (according to the subchapter 8.2). We suppose that J(x) and gi(x) are continuous and differentiable in the give domain. Let us disintegrate vector x

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (33)

into vector [x.sub.N] (of independent variables) and vector [x.sub.B] (of basic variables). Similarly, vector [nabla]

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (34)

Based on the matrix [G.sub.x], we create two sub-matrices [G.sub.N] and [G.sub.B] ; (m, n), (m, m)

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (35)

Let us indicate the reduced gradient of the cost function J(x) as [[nabla].sub.R]J [equivalent to] [v.sup.T] and its numerical calculation is carried out on the basis of a derived equation

v = [nabla][J.sub.N] - [G.sup.T.sub.N]L (36)

where [nabla][J.sub.N], [G.sup.T.sub.N], L are matrices having the dimensions (n, 1) ; (n, m) ; (m, 1). Elements of the matrix L are determined according to the equation

[G.sup.T.sub.B]L = [nabla][J.sub.B] (37)

The calculation of the values of the independent variables is performed according to [x.sup.(r+1).sub.N] = [x.sup.(r).sub.N] - h[v.sup.(r)] and according to the basic variables

[g.sub.i]([x.sup.(r+1).sub.N], [x.sup.(r).sub.B]) (38)

By this procedure, based on the allowable solution of [x.sup.(r)], we determine the values of both dependent and independent variables, thus obtaining the coordinates of the new point [x.sup.(r+1)]. The completion of the calculation is provided by the use of the criterion

J([x.sup.(r+1)]) [less than or equal to] J([x.sup.(r)]) + [epsilon], [epsilon] = [10.sup.-4] (39)

or

[parallel] [x.sup.(r+1)] - [x.sup.(r)] [parallel] [less than or equal to] [epsilon] (40)

Gradient methods belong to the iterative methods, therefore the problems of convergence have been investigated, and it is proved that [h.sub.opt] = 2/[[[lambda].sub.1] + [[lambda].sub.m]] (the convergence speed of the iteration process depends on the ratio of the matrix numbers H([x.sup.*]). The matrix H is called Hessian and has positive definite values, therefore its numbers can be arranged :0 < [[lambda].sub.1] < ... < [[lambda].sub.m]). Based on the theoretical results, we create algorithm of the generalized method of the reduced gradient. Algorithm of the generalized method of a Reduced Gradient (GMRG)

Step 1 Load the parameters values: h, [epsilon]; and the values of the independent variables vector [x.sub.N], [x.sub.N.sup.(0)] = ([x.sup.(0).sub.1], [x.sup.(0).sub.2)], ..., [x.sup.(0).sub.n]).

Step 2 The base creation and the calculation of the cost function value.

Step 3 The v direction calculation according to the relation (20) for the following iteration.

Step 4 The calculation of the independent variables values according to the relation (22) and the calculation of the set of equations in order to calculate the value of the variables of the vector [x.sub.b] (23).

Step 5 The determination of the vector x value in the r-th iteration

Step 6 In case that the values of the vector x are negative, we proceed to Step7, otherwise, we proceed to Step 10.

Step 7 We reduce the value of the Step h.

Step 8 If the value of the step h is smaller than the condition determined, in order to exchange the base, we proceed to the Step 9, otherwise, to Step4.

Step 9 We provide the exchange of the basic vectors and we proceed to the Step3, otherwise, the calculation is finished.

Step 10 We calculate the value of the cost (criterion) function in the (r + 1) iteration.

Step 11 We compare the value of the cost function in the r-th iteration with that in the (r + 1) iteration; if the value is smaller, we continue calculating, otherwise, we proceed to the Step7.

Step 12. We use a higher value of the step h and we calculate the values of the cost function in order to complete the calculation in the iteration process; in case that the condition of the required values calculation accuracy is satisfied, the calculation is finished, otherwise, we proceed to Step 13.

Step 13 Save the results of (r + 1) iteration of the x as well as the values of the cost function and continue calculating by the Step 3.

9. Generalized method of reduced gradient for the solution of discrete problems of optimum control of systems

Principal ideas of the generalized method of reduced gradient (GMRG) are discussed and main mathematical relations concerning the reduced gradient (the projection of the control vector) are formulated. An approach for the calculation of the vector of state variables and of the control vector is presented, including an algorithmic description of the GMRG procedure.

9.1 Discrete problems of optimum control of systems

The problem of optimum control is discussed for a discrete-time system which described by the mathematical model in the vector form

x(t + 1) [??] f(x(t), w(t), t), t = 0, 1, T - 1 (41)

The limitations

a(t) [less than or equal to] x(t) [less than or equal to] b(t) for t = 0, 1, ..., T (42)

[alpha](t) [less than or equal to] u(t) [less than or equal to] [beta](t) for t = 0, 1, ..., T - 1 (43)

are taken into account and the criteria function

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (44)

J(x(t), u(t)) = [f.sub.0] (x(T), T), T > [t.sub.0] (45)

are applied for a fixed value of the time T. The functions f, [f.sub.0] are continuously differentiable in respect to x(t), u(t) for any t, x(t) are column vectors of the state variables, u(t) are column vectors of the control variables, a(t), b(t), [alpha](t), [beta](t) are column vectors in the respective spaces. The problem consists in determining the admissible control vector u(t), minimizing (maximizing) the functional (45) or (46) and satisfying the conditions (42), (43) and (44). Various approximate (Hrubina, 2001) numerical methods are applicable in solving this problem of optimum control. These procedures can be divided into two groups

a) those making use of the maximum principle (Pontryagin, 1961), (Dolezal, 1981).

b) direct methods, starting from the selection of a suitable class of basic functions. The application of the generalized method of reduced gradient (GMRG) to the

problems of non-linear programming represents a natural extension of Wolfe's method of reduced gradient. A series of numerical experiments is described in, applying programmers based on the GMRG algorithm, (Hrubina, 2001). The aim of the present paper is to demonstrate the applicability of the GMRG algorithm to the direct solution of a defined problem of optimum control of systems.

9.2 Theoretical foundations of GMRG

The fundamental idea of GMRG can be successfully applied to the solution of discrete problems of optimum control in systems, such as defined by the relations (42), (43), (44) and (45) or (46). Let us designate by [psi](t) the row vector of the dual variables in relation to the vector (42) at the time t and, in a similar manner, by [omega](t), [phi](t) the dual vectors in relation to the vectors (43) and (44). The validity of the relation

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (46)

where [x.sub.N] is the vector of independent variables and [x.sub.B] is the vector of basic variables, for the reduced gradient is demonstrated in (Hrubina & Jadlovska, 2002).

The relation (47) can be expressed in the following concise form

[omega] = [f'.sub.0]([x.sub.N]) + [psi]f'([x.sub.N]) (47)

the components of the vector [psi] defining the Lagrange multiplicators.

The relation (48) can be transformed into the following relations

[omega](t) = -[psi](t - 1) + [psi](t) [[partial derivative]f/[partial derivative]x(t)], for t = 1, 2, ..., T - 1 (48)

[omega](T) = -[psi](T - 1) + [f'.sub.0](x(T)) (49)

[phi](t) = [psi](t) [[partial derivative]f/[partial derivative]u(t)] for t = 0, ..., T - 1 (50)

The Jacobian (42) contains the sub-matrices H(0), ..., H(T - 1); K(0), K(T - 1) where H(t) = [partial derivative]f/[partial derivative]x(t), K(t) = [partial derivative]f/[partial derivative]u(t) The simplest case is obtained for a(0) = 6(0), a(t) = -[infinity], b(t) = +[infinity], for all t = 1, ..., T. The basic matrix may be selected in such a manner to remain constant for any iteration. The basic vectors of this matrix are x(0), x(1), ..., x(T). The application of the examined method presumes the knowledge of at least one admissible solution x(t), u(t). The subsequent gradual substitution of x(0) = a(0) and of the given values u(t) for t = 0, 1, ..., T - 1 into the relation (1) yields the values x(t) for t = 0, 1, ..., T - 1 (direct step). This procedure of calculating the vectors for x(t) from the given values u(t) is called the Runge-Kutta Method (R.K.M.) approach (from *** http://www.emeraldinsight.com). Thus the defined control vector u(t) for t = 0, 1, ..., T - 1, as well as the state vector x(t) for t = 1, ..., T (such as calculated by means of the R.K.M. procedure) are known at the start of the iteration. The introduction of [omega](t) = 0 into the relation (49) allows the determination of [psi](t) by means of (50), (49) and of [phi](t) by means of (51) for any t = T - 1, ..., 0. For this purpose the following substitution [psi](T - 1) = [f'.sub.0](x(T)) is applied and the values

[psi](t - 1) = [psi](t) [[partial derivative]f/[partial derivative]x(t)] (51)

[phi](t) = [psi](t) [[partial derivative]f/[partial derivative]u(t)] (52)

are calculated for all t = T - 1, ..., 1 (inverse step). Moreover the value of the step [h.sub.1] of the projection of the control vector [u.sup.[omicron]](t) on the interval is determined, using the relations

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII].

After having calculated the vector [u.sup.[omicron]](t) the R.K.M. approach is applied again to determine the vector [x.sup.[omicron]](t). The step [h.sub.1] is adapted in such a manner to obtain

J ([x.sup.[omicron]](T)) [less than or equal to] J (x(T))

The iterative process is stopped when the condition

[absolute value of (J ([x.sup.[omicron]](T)) - J (x(T)))] [less than or equal to] [epsilon]

is satisfied for the positive number s, chosen in forward.

10. Conclusion

The contribution presents a mathematical model of nonlinear systems, processes control, which is expressed by a system of differential equations, constraining conditions, limitations and a defined criterion of an optimal control--a functional in the form of Bolza, Mayer and Lagrange. in general, the Lagrange optimization problem was solved, since the Bolza and Mayer problems can be converted into it. Limitations to a control u(t) and optimality conditions were stated. In order to find an optimal control [u.sup.[omicron]](t), a trajectory x(t|[u.sup.[omicron]]), the essence of the iterative gradient method was stated. The use of optimizing methods applied to the solution of control problems with limitations was also shown. The first method converts the Lagrange problem to an optimization problem of the theory of games. For searching a saddle point of a functional we present an algorithm of an iterative gradient method which can be used also in case that there are defined limitations to an optimization problem of the theory of games. For the defined mathematical model of the task of the system optimal control in a discrete shape with limitations to state and control variables, the algorithm was designed based on a generalized method of a reduced gradient (GMRG). The GMRG theory yields an effective approach not only for the establishment of an algorithm for the solution of the problems of non-linear programming but moreover for the solution of the problems of optimum contsystems, too. Numerical experiments on several defined problems were performed wit the application of the developed programmes, showing satisfactory convergence.

DOI:10.2507/daaam.scibook.2011.22

11. Acknowledgement

This work is the result of the project implementation Development of the Center of Information and Communication Technologies for Knowledge Systems (project number: 26220120030) supported by the Research & Development operational program funded by the ERDF.

12. References

Athans. M. & Falb, P. (1966). Optimal Control (An Introduction to the Theory and Its Applications). MCGraw-Hill Book Company, New York

Bellman, R. (1967). Introduction to the Matematical Theory of Control Processes I. Academic Press

Boudarel, R. & al.(1977). Commande optimal des processus. Tom 1, 2, 3, Dunod Paris

Cea, J. (1971). Optimisation, theorie et algorithmes, Dunod, Paris

Dolezal, J. (1981). On the Solution of Optimal Control Problems Involving Parameter and General Boundary Conditions, Cybernetic, Vol.17, pp.71-81

Ekeland, I. & Teman, K. (1976). Convex Analysis and Variation Problems, New York

Golder, K. & Kubik, S. (1983). Nichtlineare Systeme der Regelungstechnik. VEB Verlag Technik. Berlin

Huba, M. et al. (1999). Modern Control Theory. Progetto Leonardo. Societa Editrice ESCULAPIE, Bratislava--Roma, p. 183, ISBN 88-86524-36-6

Hrubina, K. et al. (2005). Optimal Control of Process Based on the Use of Informatics Methods, Informatech, Ltd. Kosice, p. 287, ISBN 80-88941-30-X

Hrubina, K. (2001), Algorithms of Numerical Methods and Their Application to the Solution of Optimizing Problems, Mathematical Modelling of Technical Processes, prepared within SOCRATES-ERASMUS programme, Informatech Ltd, Kosice, pp. 7-62, ISBN 80-88941-18-0

Hrubina, K. & Jadlovska, A. (2002). Optimal Control and Approximation of Variational Inequalities. Cybernetes. The International Journal of Systems and Cybernetics. MCB University Press of England. Vol 31, No 9/10, pp. 1401-1408, ISSN 0368-492X

Hrubina, K; Katalinic, B; Jadlovska, A; Wessely, E; Macurova, A & Majercak, J. (2010). Modeling and Computing Methods for Solving Optimization Problems, Chapter 42 in DAAAM International Scientific Book 2010, pp.471-488, B. Katalinic (Ed.), Published by DAAAM International, ISBN 978-3-901509-74-2, ISSN 1726-9687, Vienna, Austria

Jadlovska, A. (2000). An Optimal Tracking Neuro-Controller for Nonlinear Dynamics Systems, Elsevier Science, Pergamon, pp. 483-8. ISBN 00-08-0435467

Jadlovska, A. & al. (2005). Algorithms for Optimal Decision Making and Processes Control. Chapter 21 in DAAAM International Scientific Book. Vienna, Austria, ISBN 3-901509-43-7, ISSN 1726-9687

Jadlovska, A. (2003). The Dynamic Processes Modeling and Control Using the Neuron Networks. Scientific Writing Edition of the FEI TU Kosice, Informatech, Lt. Kosice, pp. 172, ISBN 80-88941-22-9

Jadlovska, A. & Hrubina, K. (2011). Algorithms of optimal control methods for solving game theory problems. Cybernetes. The international journal of cybernetics, systems and management sciences. Vol. 40, No.1/2, 2011, pp.290-299. Emerald Group Publishing Limited, ISSN 0368-492X,

Macura, D. & Macurova, A. (2009). Solution Systems of First Order Differential Equations by New Method, In: Chapters about Solution Differential Equations Systems and Some Applications Differential Equations, Macurova, A. (Ed.), Tribun EU, p.17-25, Brno, ISBN 978-80-7399-871-4

Pontryagin, L. S. (1961). Mathematical theory of optimal processes (in Russian), Gosenergoizdat, Moscow 1961

*** (2011) http://www.emeraldinsight.com/journals.htm?issn=0368492x&volume= 40&issue= 1 &articleid= 1921819

*** (2011) http: //www.emeraldinsight.com/journals.htm?articleid=876001

Authors' data: Assoc. Prof. PhD. Jadlovska, A[nna]*, Univ. Prof. Dipl.-Ing. Dr.h.c.mult.Dr.techn. Katalinic, B[ranko] **, Assoc. Prof. PhD. Hrubina, K[amil] *; EdDr.PhD. Macurova, A[nna] *; Assoc. Prof. CSc. Wessely, E[mil] ***, *Technical University of Kosice, Letna 9, Kosice, Slovakia, ** University of Technology, Karlsplatz 13, 1040, Vienna, Austria, *** University of Security Management in Kosice, Slovakia anna.jadlovska@tuke.sk, katalinic@mail.ift.tuwien.ac.at, kamil.hrubina@tuke.sk, anna.macurova@tuke.sk, emil.wessely@vsbm.sk.

Tab.1 Classification of programming problems

Criterion Contraints x Type of the problem
 function g(x), h(x)
 J(x)

1. linear linear Linear programming
2. linear linear integer Integer programming
3. quadratic linear Quadratic programming
4. convex convex Convex programming