THE UNIVERSITY OF MICHIGAN COLLEGE OF LITERATURE, SCIENCE, AND THE ARTS Department of Mathematics Technical Report No. 3 STATEMENT OF PONTRYAGIN'S NECESSARY CONDITION FOR LAGRANGE PROBLEMS, HAMILTON-JACOBI EQUATION, AND FIELDS Lamberto Cesari ORA Project 010582 submitted for: UNITED STATES AIR FORCE AIR FORCE OFFICE OF SCIENTIFIC RESEARCH GRANT NO. AFOSR-71-2122 ARLINGTON, VIRGINIA administered through: OFFICE OF RESEARCH ADMINISTRATION ANN ARBOR November 1971

-~~~~~~~~-;"-~e Up-~ } 4: an*~ "ru *W- al

3.1 35. STATEMENT OF PONTRYAGIN'S NECESSARY CONDITION FOR LAGRANGE PROBLEMS, HAMILTON-JACOBI EQUATION, AND FIELDS 3.1 The statement As we have shown in (1.1), Lagrange, Bolza, and Mayer problems can be essentially transformed one into the other very easily by suitable change of space variables. In particular, Lagrange and Bolza problems can be written in an equivalent Mayer form, and then Pontryagin's necessary condition (2.2.i) can be applied to the Mayer problem so obtained. Nevertheless, for Lagrange problems, there are some simplifications, which make it worthwhile to reformulate Pontryagin's necessary condition specifically for problems written in the Lagrange form. We consider here a Lagrange problem concerning the minimum of the functional I[x,u] = J f (t,x(t),u(t))dt, f scalar, (31.1) t0 with x(t) = (x,...,x ), u(t) = (u,...,u ), satisfying the differential system dx/dt = f(t,x(t),u(t)), f = (f1,.' nf )' (3.1.2) boundary conditions (tl,x(tl),t2,x(t2)) C B c E2n+2, and constraints (t,x(t)) c A, u(t) c U. Here all n+l scalar functions fi(t,x,u), i = 0,1,...,n, are assumed to be defined and continuous in the set M of all (t,x,u) with (t,x) C A, u C U(t) (or M = A x U if U is a fixed set), together with their partial derivatives

3.2 fit f i = 0,1,... n, j = 1 n. The class Q of the admissible pairs it' ix''' x(t),(t) t), t < t < t2, is defined as in (1.1), though we must now require explicitly that f (t,x(t),u(t)) be integrable in [tl,t2]. As in (2.1) we assume that x(t), u(t), t1 < t < t2, is an optimal pair and we need not repeat the assumptions (abcd) of (2.1) for the situation under consideration. In particular, the graph of the optimal trajectory x is in the interior of A, U is a, fixed set (bounded or not), and the optimal strategy u is bounded. (Alternatively, assumptions corresponding to c', c", etc., can be made.) We shall need now an (n+l)-vector a = (k0,nl,. t,), and the Hamiltonian function H(t,xu,X) = 0 f0(t,x,u) + klfl(t,x,u)+...+ f n(t,x,u). (3.1.3) Also, we shall need the related function M(t,x,X) = Inf H(t,x,u,X). (3.1.4) ucU Pontryagin's necessary condition for Lagrange problems take now the form below, which we shall deduce explicitly from (2.2.i) at the end of this section. (Pi') There is a, continuous (AC) vector function X(t) = (x0,kl,..,f ), tl < t < t2 (Pontryagin's multipliers) which is never zero in [tl,t2], with Xg a, constant in [tl,t2], 0 > 0, such that di /dt = -H i(t,x(t,x ( t)), i = 1,...,n, x for t in [tl,t2] (a.e.).

3.3 (P2') For every fixed t in [t,t2] (a.e.), the Hamiltonian H(t,x(t),u,X(t)) as a. function of u only with u in U takes its minimum value in U at u = u(t), or M(t,x(t),7(t)) = H(t,x(t),u(t),)(t)), for t in [tlt ] (a.e.). 12 (Pr') The function M(t) = M(t,x(t),(t)) is continuous (AC) in [tl,t2] and dM/dt = dM(t,x(t),\(t))/dt = -Ht(t,x(t),u(t),X(t)), for t in [t,t2] (a.e.). (P4') (Transversality relation): n n -M(t )dt + Z x (t )dx + M(t2)dt - Z x (t )dxJ = 0 1 1 j1j 2 1 2 j 2 j=l j=l for every vector h c B', or (2.1.3), that is, M(t)dt - j (t)dx = 0. (3.1.5) j=l J 1 The transversality relation is identically satisfied if tl,xl,t2,x2 are fixed, that is, for the boundary conditions which correspond to the case that i i "both end points and times are fixed" (dtl = dx = dt = dx = i = 1,.,n). Here x,u is an admissible pair itself so that the differential equations dxi/dt = fi(t,x(t),u(t)), i = 1,...,n, (3.1.6) hold and these equations, together with (P1), yield the canonic equations

3.4 i dk dx 1H - 1,H da di i = 1,... (,n.3(17) dt dt These same equations as well as (P3) can also be written in the explicit forms dx /dt = fi(t,x(t),u(t)), i =, r, n dki/dt = - E j(t)f.(t,x(t),(t)), i =. n, dM/dt = - Z \(t)tf (t, x(tU(t)), t < t < t2 j=O jt j=0 i jt 1 -0 - If we denote by x~ an auxiliary variable satisfying the differential equation dx 0/dt = f (t,x,u) and initial condition x (tl) = 0 then t2 0 0 0 I[x,u] = f f0dt = x (t ) x (t2) = x (t2) t] and the two equations dx H 0 _ H dt \0' dt a' can be added to the canonic equations (3.1.7), since aH/&ko = fo' and H 0 = 0, x and hence A0 is a constant, as stated. Finally, let us note that for autonomous problems, that is, for problems where fO, fl,...,f do not depend on t, but only on x and u, we have fit = afi/6t = 0. Hence, (P3')yields dM/dt = 0, and the function M(t) is a constant in [tl,t2]. In particular, for autonomous problems between two fixed points x (x=, i = 1,...,n) and x2 = (x2, i = l,..,n), in an undeterminate i tine, then we can take t1 = 0, dxi dax = 0, i = 1,...,n, dt2 arbitrary.

35.5 Hence, (P4') yields M(t2)dt2 = O, or M(t2) = 0, and M(t) is the constant zero in [tl,t2]. This occurs, for instance, in a problem of minimum time (between two fixed points), where fo = 1, and then I = t2. 3.2 Derivation of properties (P'l-4) from (P1-4) We shall now deduce (P'l-4) from the analogous relations (P1-4) of (2.2). To this purpose let us first transform the Lagrange problem above into a Mayer problem. As mentioned in (3.1), we introduce an auxiliary variable x, the n+l extra differential equation dx /dt = f (t,x(t),u(t)), and the extra boundary condition x (tl) = O. Thus t2 n+l n+l n+l I[x,u] = f f dt = x (t) x (t) = x (t). 0 2 1 2 tl We denote now by x the (n+l)-vector x = (x,...,x,x ), and by f(t,x,u) the (n+l)-vector function f(t,x,u) = (fl,...'f, f f)' We have now the Mayer problem of the minimum of the functional n+l J[x,u] = g = x (t2) = I[x,u], with differential equations n+l dx/dt f /dt = f (t,xu), i =,...,n, /dt = f(t u), n+l boundary conditions (tl,x(tl),t2,x(t2)) c B, x (tl) = O, and the same constraints as before (t,x(t)) c A, u(t) c U(t,x(t)), where A = A x E1. The boundary conditions can be expressed in the equivalent form (tl,x(tl),t2,x(t2)) C B where B is the closed subset of the space E2+4 defined by 2n+4

3.6 n+l n+l B = B x (x = O) x (x E ), since we have assigned the fixed value n+l n+l n+l n+l x1 = 0 for x (tl), and we have left undetermined the value x = x (t2). n+l n+l Thus, in any case, we have dxl = 0, and dx is arbitrary. In other words, ~1 ~2 the new set B' is the set B' = B' x (0) x (E1) since dx2 is arbitrary. According to (2.2), we need now an (n+l)-vector A = (l,...,kn*A +)) the Hamiltonian function H(t,x,u,X) = f1 +.+ f + n+l fO and the related function M(t,x,%) = Inf H(t,x,u,X), uEU where we have written x instead of x since these functions do not depend on n+l x. Also, we shall need nonnegative constant k0 0. 0 - We are now in a. position to write explicitly statements (P1-4) of (2.2.i), for the so obtained Ma.yer problem, where n+l replaces n of (2.2.i). The following remarks are relevant. First, from (P1) of (2.2.i), we obtain for the multiplier A the equation d n+l/dt =H where now Hn+l 0 hence x x dX +l/dt = 0, and x +l(t) is a constant in [t,t2]. Secondly, let us write (P4) for a. very particular case of the new problem, namely, when dt =dx = dt = dx = i,...,n, dxl = 0 and dx is arbitrary. 1 1 2 2i' 1 2 n+l n+l Here g is the function g(tl,xl, t2 x 2, and hence gxn+l = ag/x2 =i, while all the other partial derivatives of g are zero. Then (P4) reduces to n+l n+l the simple relation [k0 - n+l(t2)]dx2 = 0 with dx2 arbitrary; hence kn+l(t2) = Xg, and since +l(t) is a constant we have +l(t) = a, a

3.7 nonnegative constant in [tl,t2]. We can now identify the multiplier n+l with the constant 0. Then the (n+l)-vector x can be written in the form = (Al',...,n,)0), and H becomes H = X0f0 + Xlfl +...+ f, that is, the Hamiltonian H takes the form (3.1.3). For this new problem, with the above notation, (P1-4) now become the statements (P'l-4) listed above. The auxiliary variable x should now be denoted by x, as mentioned at the end of statements (P'l-4). 3.5 Examples of application of Pontryagin's necessary condition for Lagrange problems (a) Exercise 1 of (2.3) can be written as the Lagrange problem of the minimum of the integral t2 I[x,y,u] = J dt = t2 t tl (where we can take t = O), with differential equations, constraintS and boundary conditions dx/dt = y, dy/dt = u, u U = [-l < u < 1], x(tl) = a, Y(tl) = b, x(t2) = O, y(t2) = 0. Then fo = 1, fl = y, f2 = u, and the Hamiltonian is H = 0 1y + y + 2u, with Xg a constant. The analysis is now the same as for exercise 1 of (2.3), and the optimal time t2 is the same. Note that here the Hamiltonian is H = g +,y + 2U with, n1stant, and k1,A2 satisfying dl/dt = 0, d\2/dt = -Al, hence 1 =1'

3.8 2 =-ct + c2, cl,c2 constant, as for exercise 1 of (2.3). Again here Ht = 0, dM/dt = 0, and M is a constant. On the other hand, dt1 0, dx = dy = dy = dy = O, dt2 arbitrary, and (P4') yields M(t )dt = O, or M(t2) = O. Thus, M(t) is the constant zero in [tl,t2], as stated in general at the end of (3.1). We can take XO = 1, and cl,c2 as stated in the Remark after exercise 1 of (2.3). (b) The problem of the curve of minimum length between two points 1= (tl,xl), 2 = (t2,x2), tl < t2, in the tx-plane, was written in exercise 5 of (2.3) as the Lagrange problem of the minimum of the functional t2 I[x,u] = 2 (l + u2(t))l/2dt tl with differential equation and boundary condition dx/dt = u, x(tl) =x x(t2) = x. Then f = (l+u )/ f = u, and the Hamiltonian is H(t,x,u,X) = 0 (l + u2)1/2 + lU, with k > 0 a constant, and k1 satisfying d>l/dt = 0, hence kl = cl, a. constant. Here H has a minimum as a function of u in -a< u < +x, if k 0> k J), and for the minimum we must have u(l +u 2)l1/2 + X = 0, where both X and k1 are constants. Thus, lu is a. 0 1 0 1 constant c in [tl,t2], x is linear, and we obtain the usual segment s = 12, as in exercise 5 of (2.3). Concerning the remaining consideration, again we have Ht = 0, dM/dt = 0, and M(t) is a constant. On the other hand, tlxlt2,x 2 are fixed, and (P4') is identically satisfied. We can take X = 1 and = -c(l+c2)-l/2 as in exercise 5 of (2.3).

3.9 3.4 Examples of transversality relations for Lagrange problems We shall now apply transversality relation (P4') of Pontryagin's necessary condition for Lagrange problems (3.1) to a, number of particular but rather typical cases. We restate here that x(t), u(t), tl < t < t2, is an optimal pair, and that, therefore, by (P2'), (P3'), M(t) = M(t,x(t),X(t)) = H(t,x(t),u(t),%(t)) n = X f (t,x(t),u(t)) + Z ki(t)fi(t,x(t),u(t)) i=l for t in [tl,t2] (a.e.). (3.4.1) Also we mention here that, for autonomous problems, that is, fo(x,u),...,f (x,u) depending on x and u only, M(t) is constant in [tl,t2]. (a) As mentioned in 3.1, in the case of both end points and times fixed, or 1 = (t ) 2 = (t 2,x2) both fixed, transversality relation (P4') is trivial, i i dt = dx = dt = dx2 = O, i = 1,..,n. 1 1 2 2 1.., (b) Let us consider the case concerning first end point 1 = (tl,x l) fixed, second end point 2 = (t2,x(t2)) on a, given curve r: x = b(t), t' < t < t", x hence x(t2) = b(t2), where b(t) = 2 2F xlI ~ 1 of (bl,...,b) (case (b) of 1.6). Let us assume that 2 is not an end point of r, t ti t 1 2 that is, t' < t < t" and that b is differentiable at t2. By the same argument as in (2.4) we obtain the (finite) transversality relation

3.10 M(t2) - ii(t2)b(t 2) = 0. (3.4.2) (c) Let us consider the case concerning first end point 1 = (tl,xl) fixed, second end point x2 fixed in E (target), to be reached at some undetermined x time t > t1 (case (c) of (1.6)). By X2 > the same argument of (2.4) and the use x, --- I of (P4') (instead of (P4)), we obtain tl t the unique (finite) transversality relation M(t2) = 0. (3.4.3) Thus, for autonomous problems, M(t) is the constant zero in L[t,t ]. (d) Let us consider the case concerning first end point 1 = (tl,xl) fixed, second end point x2 on a given set S in E (target), with S to be reached at some undetermined time t > t (case (d) of (1.6)). Let us consider first the 2- 1 x2 case, say (dl), where S = E is the x n 1 whole x-space, and assume t > t. (In x 12 2 1 at X 2L__, At the illustration, we have taken n = 2.) The same argument as in (2.4), and the use of (P4') (instead of (P4)), yields the n+l (finite) equations M(t2) = 0, <i(t2) = 0, i = 1,...,n. (3.4.4) Let us now consider the case, say (d2), where S is a given curve in E, say S: x = b(T), T' _ < T, T a parameter, b(T) = (b1(),...,bn(m)), and

3.11 assume that x hits the target S at a x ~ x I S x~time t2 > tl (as above), and at a point! -"' L ) t x2 which is not an end point of S, precisely x2 = x(t2) = b(T0) for some TO, T' < To T?". The same argument as in (2.4) yields the two (finite) equations n M(t) = 0, Z (t )b'() = 0. (3-4.5) 2 j= j 2 j The second relation states that the vector Xi(t ), i = l,...,n, is orthogonal at 2 to the line I which is tangent to the curve S at the same point. Let us consider now the case, say (d3), where S is a given k-dimensional manifold in E. If we assume that S is a given parametrically as before, say S: x = b(T), b(T) = (bl,...,b ), T = (Tl'''''Tk), then x2 = x(t ) = b(T0) for some To = (T10''..' k0) We assume that b(T) is defined in a neighborhood V of TO and that b(T) is of class C' in V. The same reasoning a.s in (2.4) yields the k+l (finite) equations n M(t2) = 0, Z Xj(t2)bjs (T0) =, s = l,...,k. (3.4.6) j=l Again the last k relations state that the n-vector xi(t2), i = l,...,n, is orthogonal at 2 to the tangent plane I to the manifold S at the same point. If S is given by n-k equations f (x) = O, a = l,...,k, and then n dx2 = (dx2,...,dx2) is any vector satisfying Zj.(x29 dx~ = O, a = l1,...,n-k, (3.4.7) then, by the same reasoning as in (2.4), we see that (P4') yields M(t2) = 0,

3.12 as well as n Z j (t2)dx2 = 0 (3.4.8) j 2 2 j=1 for all dx2 = (dx 2,...,dx2) satisfying (3.4.7). Again the latter restates that the n-vector ki(t2), i = 1,...,n, is orthogonal at 2 to the plane ~ which is tangent to S at the same point. For autonomous problems, in all these cases (dl,2,3), M(t) is the constant zero in [tl,t2]. (e) Let us consider the case concerning first end point 1 = (tl,x1) fixed, terminal time t2 fixed, t2 > tl, and terminal point x2 on a given set S in E (target) (case (c) of (1.6)). Let us consider first the case, say (el), where S = E is the whole x-space. In other x n 1 X 2 words, the target S is a fixed hypert plane t = t2 in the tx-space En+1 In the illustration we have taken n = 2. The same argument of (2.4) and the use of (P4') (instead of (P4)), yield the n (finite) equations ki(t2) = 0 i = l,...,n. Let us consider the case, say (e2), where S is a given curve in E, say n'e n L2 that x hits the target S (at fixed time 1/ S t en t2) at a. point x2 which is not an end point of S, precisely, x2 = x(t2) = b(To) 2 for some T0, T' < T~ < ". The same

3.13 argument of (2.4) and (P4') (instead of (P4)) yield the only (finite) equation n Z xj(t2)bj(To) = 0, (349) j=l i i and this relation states that the n-vector ki(t2), i = l,...,n, is orthogonal at 2 to the line which is tangent to the curve S at the same point 2. Let us consider now the case, say (e3), where S is a given k-dimensional manifold in E. If we assume that S is given parametrically as before, say n S: x = b(T), b(T) = (bl(T)*...,bn(T)), T = (T1,,Tk~' then x2 = x(t = b(T ) for some To = (T10'..' Tk0). We shall assume that b(T) is defined in a neighborhood V of T and that b(T) is of class C' in V. The same argument as in (2.4) and (P4') (instead of (P4)) yield the k (finite) equations n Z %j(t2)bj() =, s = 1,...,k. (34.10) j=l Again these equations state that the n-vector i (t2), i = l,...,n, is orthogonal at 2 to the tangent plane X which is tangent to S at the same point. Finally, if S is given by n-k equations Pi (x) = 0, a = l,...,n-k, and then dx2 = (dx2,...,dx2) is any n-vector satisfying equations (3.4.7), then the same argument as in (2.4) and (P4') (instead of (P4)) yield that the relation n j X(t)dx = (34.11) j= 2 1 n hold for every dx2 = (dx2,...,dx2) satisfying (3.4.7). Again this is a restatement that the n-vector \j(x2), j = l,...,n, is orthogonal at 2 to the tangent plane ~ to S at the same point.

3.14 We finally remark that the assumption that the pair x,u be optimal, that is, I[x,u] has an absolute minimum at x,u in Q, can be replaced by the assumption that the pair x,u is a strong relative minimum, that is, I[x,u] < I[x,u] for all pairs x,u whose trajectory x satisfies the same boundary conditions as x and lies in any given small neighborhood N5 of x. 3.5 Hamilton-Jacobi partial differential equation We shall consider here a domain V of the tx-space, V c A, and V may be smaller than the original subset A. We shall consider the Lagrange problem of the minimum of the functional t2 I[x,u] = J f0(t,x(t),u(t))dt (3.5.1) t1 with side conditions and constraints dx/dt = f(t,x(t),u(t)), f = (fl'''f) (.3.52) (t,x(t)) E V, u(t) E U, (3.5.3) in the class Q of such pairs x(t), u(t), t < t < t2, transferring any given point 1 = (tl,xl) to a given target S. If the minimum of I[x,u] in n exists, x and X (t), uo(t), tl < t < t2I is any V optimal pair, then w (tlx1) = I[xO u II I - tl t 2 Min I[x,u] (x, u)cEQ does not depend upon the particular optimal pair xouO. Instead, we shall consider here w~hLt Hippens when we vary the initial point (tl,xl).

3.15 First we shall consider all points (tl,x1) for which the problem above has a minimum. Let 3 = (t,x) be any point of an optimal trajectory xO(t), uO(t), t1 < t < t2, say x = xo(t), t1 < t < t2. Clearly the point 3 = (t,x) can be transferred to S in V, since the pair x (t), Uo(t), t < t < t2, that is, the restriction of xO, U0 to the subinterval [t,t2], evidently performs the task. Thus, the class Q of all (admissible) pairs x,u transferring 3 = (t,x) to S is not empty. (3.5.i) (Property of optimality). Every point (t,x) of the trajectory x0, say x = x(t), t < t2' is transferred to S in V optimally by the pair x (t), uo(t), t < t < t2. Proof. Let 1 = (t1,x1), 2 = (t2,x(t2)). If the statement were not true, then there would be another admissible pair xl(t), u (t), t t < t t2, transferring 3 = (t,x) to some point, say 4 = (t',xl(t )) on S. with, briefly, I34 < I. poi, 2' 1 2' Let us consider now the admissible pair X(t), v(t), t < t < t' defined by 1- 2' X = xO, v = uO on tl,t], X = x1, v = u1 on [t,t]. Then X,v transfers I2 1 1' 1) 14 1 34 13 322 since xO,u0 transfers 1 to 2 on S optimally. This proves (3.5.i). We shall write here the Hamiltonian for the Lagrange problem above as defined by (3.1.3) with the constant multiplier k0 equal to one, or n H(t,x,u,X) = fo(t,x,u) + kifi(t,x,u). (3.4) i=l Let x(t), u(t), t < t < t2, be any admissible pair transferring the point 1 = (tl,x1) to a target S in V. Thus xl = x(tl), (t,x(t)) E V, u(t) U, t1 < t < t2. Let us consider any point 3 = (t,x(t)) of the trajectory x, and let us take 3 to S along the same trajectory x, that is, by the admissible pair x(T), u(T), t < T < t2. Then the corresponding value of the functional (3.5.5) below is a function of t in [tl,t2].

3.16 (3.5.ii) If there exists a function c(t,x) of class C in V such that t2 W(t,x(t)) = f (T,x(T),u(T))dT, t1 < t < t2, (35 5) t then wt(t,x(t)) + H(t,x(t),u(t),c (t,x(t))) O, t < t < t2. (3.5.6) Proof. Since w(t,x) is of class C1 in V and x,u is admissible, we have n dw(t,x(t))/dt = wt(t,x(t)) + w. i(t,x(t))dx /dt t ~~x i=l X n = t(tx(t)) + i(t,x(t))fi(t,x(t),u(t)). i=l (3.5.7) On the other hand, by force of (3.5.5), we have dw(t,x(t))/dt = -f (t,x(t),u(t)) and by comparison with (3.5.7) we have n t(t,x(t)) + f0(t,x(t),u(t)) + Z xi(tx ( ))fi(t)) 0. x i (3.5.8) In view of (3.-.4),this relation is exactly relation (3.5.6). Here Wt' xi) i = 1,...,n, are the first order partial derivatives of o, and w denotes the x n-vector (ot 1,'''c. ), X Xx We shall assume below that V is simply covered by a family of optimal trajectories x(t), t < t < t2, that is, for each point (t,x) E V, there is a well determined pair x(t), u(t), t < t < t2, with x(t) = x, t t < t2, 1- 2'' 1- -2'

3.17 transferring optimally (tl,xI = x(tl)) to the target S with (t,x(t2)) C S, and hence by (3.5. i), transferring optimally (t,x) to S. In this situation the x values u of the strategies u(t) become V functions of (t,x) in V by means of the S equations u(t,x) = u(t), x = x(t), (t,x) C V. Also, a function w(t,x) is defined in V by means of the equations t2 wc(t,x) = f f (T,x(T),u(T))dT, x = x(). -0' Thus t2 w(t,x(t)) = f f o(,x(T),u(T))dT, t1< t < t t for each trajectory x(t), t1 < t < t2, of the family covering V. Here 4(t,x) is the optimal cost in transferring (t,x) to S in V. (3.5.iii) Ifw(t,x) is of class C in V, then X satisfies the Hamilton-Jacobi partial differential equation wt(t,x) = -H(t,x,u(t,x),c (t,x)), (t,x) E V. (3.5.9) Also, on each trajectory in V we have (d/dt)co i(tx(t)) = -fgi(tx(t),u(t)) n - Z w a(t,x(t)fai(t,x(t),u(t)). (3.5.10) ~1= X

3.18 In other words, ki(t) = X i(t,x(t)), i = 1,...,n, t < t < t are Pontryagin's x 1multipliers of the trajectory x(t), t < t < t2) in V. Proof. From (3.5.ii)weknow that (3.5.9) is satisfied everywhere in V. For every point (t,x(t)) E V and any fixed v E U. let us consider the constant control function u(T), t < T < t +At, with constant value u(T) = v, and let x(T), t < T < t +At, the corresponding trajectory with initial value x(t) = x(t). The trajectory x certainly exists is uniquely defined, and lies in V provided At > 0 is sufficiently small. Then x = x(t) is taken by u to a point x(t) + Ax = x(t +At), with Ax = (Ax,...,Ax ), and Ax = f.(t,x(t),v) At + E.At, i = l,...,n, (3.5.11) where Ei + 0 as At * 0. Indeed x is certainly continuously differentiable in [tt + At], and Ax/At + x't) = f(tx= f (t),v) as At - 0. Thus'x /At - f.(t,x(t),v), i = 1,...,n, as At -> 0, and (3.5.11) follows. The 1 corresponding cost in transferring x = x(t) to x(t) + Ax is then given by the integral (3.5.12) below. By the continuity of f0 and x, we have t+At f f0(Tx(T),v)dT = f0(t,x(t),v) At + o 0At, (3..12) t where E0 - 0 as At * 0. Now the point x(t) + Ax can be transferred to the target S optimally with a. cost cu(t + At,x(t) + Ax), and by Taylor formula A = c(t +At,x(t) +Ax) - co(t,x(t)) n = t(t,x(t))At + Z x a(t,x(t))Axa + c'At, ~t~ ~ X a=l x

3.19'v:<hera r - a 0 as At -- O. By force of (3.5.11), we have now A = t(t,x(t)) + E cue(t,x(t))f (tx(,t),v) At + c"At 7where c" + as At - C. Now, by force of (3.5.i) the cost of transferring (t,x(t)) of S via. (t +At,x(t) + x) is certainly > c(t,x(t)), or f (t,x(t),v)At + EoAt + a(t +At,x(t) +Ax) > c(t,x(t)). By the use of (3.5.13), we have now Kfo(t,x(t ),v) + wt(t,x(t)) n + E W a(t,x(t))f (tx(t),v) At > (-0- c")At, x where c0, E'! + 0 as At - 0+. By dividing (3.5.15) by At > 0 and taking the limit as At + O, we obtain n f (t,x(t),v) + et (tx(t)) x(t))f(t,x(t),v) > O (3.5.16) for every v c U and every (t,x(t)) c V. Let g(t,x,u) denote the function n g(t,x,u) = fo(t,x,u) + wt(t,x) + ZE u U(t,x)f (t,x,u), where (t,x) e V, u c U. By comparison with (3.5.8) and (3.5.15), we see that g(t,x,u) > 0 for all (t,x) e V, v E U, and that g(t,x(t),u(t)) = 0. This can be interpreted by saying that, for u = u(t), the function g(t,x,u(t)) has a minimum at x = x(t). Since (t,x) is an interior point of V and g possesses first order partial derivatives g i(t,x,u), i = l,...,n, we conclude tnat x

3.20 gXi''Ix=x(tu=u(t)n or n f ijt,x(t),u(t)) + ct (tx(t)) + Z CXa i(tx(t))f (t(t),x (t),(t)) Ox tx c1 xx n + Z co c(t,x(t))f i(t,x(t),u(t)) = 3, i = 1,...,n. (315.17) Here f - dx /dt, a = l,...,n, hence the sum of the second and third terms in (3.5.17) is (d/dt)wc i(t,x(t)), and (3.5.17) becomes x n (d/dt)w i(tx(t)) = -f i(tx(t),u(t)) - E X (tx(t)) X Ox x f i(t,x(t),u(t)), i =,...,n. ax Statement (3.5.iii) is thereby proved. Examples. (a) Let us consider the same problem of minimum time of example (a) of (3.3), that is, the same exercise 1 of (2.4) with the functional t2 written in Lagrange form. We have seen in example (a) of (3.3) that the Hamiltonian is H = Ad / + y + 2u, where we -an take O 1, 1 =x k2 =' X Thus, equation (3.3.9) reduces to 0 = l+yc +uc+. Comparing with exercise 1 x y of (2.4), we know that u = -1, or u = 1 according as (x,y) is above or below the switch line AOB. Thus, the Hamilton-Jacobi partial differential equation above the switch line is ct +Ycx - o +1 = 0 t x Y

3.21 and below the switch line is ct + yj) + ) + 1 = 0. t x Y We can verify easily that the optimum time w(x,y) given by u(x,y) y + 2(2-1 y+ x) 1/2 for (x,y) above AOB, and w(x,y) = -y + 2(2 y - x)/2 for (x,y) below AOB, certainly satisfy the Hamilton-Jacobi equation. (b) Let us consider now problem (b) of (3-3), that is, the path of minimum length between two fixed points 1 = (tl,x1), 2 = (t2,x2), t < t2. We have seen there that the Hamiltonian is H = k (l + u2)/2 + klu. Here we can 0 1 take = 1, X = Xk so that equation (3.3-9) becomes wt + (l+u2 1/2 +W u = 0. x 1 t x On the other hand, we have seen in example (b) of (3.5) that the relation be2 -1/2 tween u and hi (for XO = 1) is u(l+u )1/ + Al = O. Thus, u(l +u ) + x = 0; hence, -1 < < < 1, c and u have opposite signs, and x x x solving with respect to u we have 2 -1/2 2 2-1 u = - (l - co), + = (1- ). x x x By substitution, we find ot + (l- 2)1/2 = 0, and finally the Hamilton-Jacobi x equation is 2 2 + X = 1. "t x It is easy to verify that, for instance, o(t,x) = ((t-a)2 + (x-b)2)1/2, for a,b fixed, certainly satisfies this partial differential equation.

3.22 3.6 Fields We shall consider here a domain V as above, and we assume that (a) There exists a family (x(t),u(t), tI < t < t 2 of optimal pairs, and corresponding Pontryagin multipliers K(t) = (kl,'.., ), each pair x,u of the family transferring its initial point 1 = (tl,xl) to a given target S, with trajectories x lying in V and filling V once. By this we mean first of all that for each point (t,x) e V-S there exists one and only one pair x(t), u(t), t1 < t < t2, of the family with t < t < t2 x = x(t). Also, we assume that each pair x(t), u(t), t 1< t < t2, being optimal, satisfies Pontryagin's necessary condition, with multipliers x(t), so that necessary conditions (P'1-4) of (3.1) are actually satisfied. Under the assumption (a), then, for each point (t,x) e V-S there is a well determined system x(t), u(t), K(t), t! < t < t2, with x = x(t), t < t < t2, and hence a well determined m-vector u(t) which we shall denote by u(t,x), and a well determined n-vector A(t) which we shall denote by %(t,x). Thus, we have two functions u(t,x), k(t,x), (t,x) c V. In other words, to each point (t,x) E V there is assigned a well determined control u = u(t,x), and the Pontryagin multipliers k(t,x). Obviously, the two examples (a), (b) of (2.3) lead to such a situation. It is said that a control synthesis has been performed in V. Very often, in particular in the two examples mentioned above, such a synthesis can be performed in the whole tx-space E (or at least in the whole subset A of this space in which the problem has been defined in the

3.23 first place), each pair x(t), u(t) being the unique optimal pair transferring each point (t,x(t)) to S optimally in E (or A). However, it may well occur n+l that a synthesis can be performed (if any) only in a rather small part V of E (or A), and there is no guarantee that an analogous synthesis can be performed in some set V' larger than A, or that the optimal trajectories x filling V are still optimal when we enlarge V. Note that, a, synthesis having been performed in V, the cost functional w(t,x) to transfer optimally (t,x) to S in V is given by t2 (zt,x) = f f0(t,x(t),u(t))dt, t where x(t), u(t), t < t < t2, is the only pair of the given family with 1 2 t < < t, x = x(). Thus, we have now three functions u(t,x), k(t,x), w(t,x) defined in V. Having performed a synthesis in the region V as above, we shall say that V is a field provided (b) u(t,x) is of class C1 in V-S, and k(t,x), c(t,x) are of class C2 in V, and X(t,x) = X (t,x), or %i -= i. i = 1,...n. x i x We shall also assume, as usual, that f (t,x,u),...,f (t,x,u) are continuous functions in (t,x,u), with continuous first order partial derivatives f in iJ O0,l,...,n, i =,..,n. jx A first property of a field (under hypothesis (a)) follows from Pontryagin necessary conditions (P'1-4), and principle of optimality (3.5. i). Indeed,

3.24 each point (t,x) C V is transferred optimally to S in V by solving the differential system and initial condition dx/dt = f(t,x,u(t,x)), x(t) = x. Indeed now, this system has a unique solution satisfying the initial condition x(t) = x, and this solution must, therefore, coincide with the given trajectory x(t) since dx(t)/dt = f(t,x(t),u(t)), u(t) = u(t,x(t)). Also the Pontryagin multipliers X(t) = (1!,...,n) already associated with the optimal pair x,u satisfies the differential system di/dt = -Z... f.(t,x,u(t,x)), \(t) = X(t,x). 1 J J jxi' Indeed, now this system has a unique solution satisfying the initial condition A(t) = X(t,x), and this solution must, therefore, coincide with the given system A(t) of Pontryagin multipliers since dX(t)/dt = -j fji(t,x(t),u(t)), u(t) = u(t,x(t)). By Pontryagin's necessary condition (P3'), we have now H(t,x,u(t,x),\(t,x)) = Min H(t,x,u,\(t,x)), (t,x) C V, uEU or, by force of (b), also H(t,x,u(t,x),w (t,x)) Min H(t,x,u,c (t,x)), (t,x) e V-S. ucU

3.25 Also, by force of (3.5), we know that w(t,x) satisfies the Hamilton-Jacobi equation co (t,x) + H(t,x,u(t,x),c (t,x)) = 0, (t,x) C V-S. t x Problems 1. We have seen that the Hamilton-Jacobi partial differential equation of the 2 1/2 integral length (1 +x' ) / dt is V2 + V2 1 t x (a) Verify this equation for a field of parallel straight lines. (b) Verify this equation for a field of straight lines converging to a point. 2. Write exercise 2 of (2.3) as a Lagrange problem (f = 1), and write the corresponding Hamilton-Jacobi partial differential equation. 3. Same as number 2 for exercise 3 of (2.3). 3.7 A sufficient condition for Lagrange problems We shall consider here a domain V of the tx-space En, and the Lagrange problem of the minimum of the functional t2 I[x,u] = I fO(t,x(t),u(t))dt, t1 with side conditions and constraints dx/dt = f(t,x(t),u(t)), f = (fl''''fn)' (t,x(t)) e V, u(t) C U,

3.26 in the class i of pairs x(t), u(t), 1 < t < t, transferring a given point tl — 2' X t ~~xl t 0~ ~ 1 = ~(tlx1) c V to a target S c V. ~~~~~~I ~Let H(t,x,u,%) be the Hamiltonian t' *~~~~~~t of the Lagrange problem above which we shall write with the multiplier k0 = 1: n H(t,x,u,) = fo(t,x,u) + E kifi(t,x,u). i=l We shall assume that (a) For each (t,x) C V and x E E, the function H(t,x,u,%), as a funcs tion of u in U, has a unique absolute minimum at u = u(t,x,%). H(t,x,u(t,x,k),k) < H(t,x,u,k) (3.7.1) for all u C U, u f u(t,x,%), and for all (t,x) E V, x c E. (b) There exists a function w(t,x) of class C in V satisfying the partial differential equation Ct(t,x) + H(t,x,u(t,x,wx(tx)),W (tx)) = O, (3.7.2) where /t = x/at, (1''J x n) is the n-vector of the first order parx x x 1 n tial derivatives of w with respect to x,...,x; and o(t,x) = O for (t,x) c S. (c) A pair xo(t), uo(t), tl < t < t2, in the class Q, has the property

3-27 Uo(t) = u(tX (t),x(t,Xo(t))). (3.7.3) (3.7-i) Statement. Under hypotheses (a), (b), and (c) we have I[x,u ] < I[x,u] for every pair x,u in Q. Proof. Let x(t), u(t), t < t < t3, be any pair in Q. Thus x,u, as well as XOUO, transfer 1 = (tl,xl) to S in V, hence x1 = x (t ) = (tl)Y (t 2,x(t2)) S, (t,x(t )) c S. By hypothesis (b) we have wt(tx) + fo(t,x,u(t,x,a (t,x)) n + W i(tx)f (t,xutxux (tx))) 0 (3.7.4) i=l for all (t,x) E V. By taking x = x (t) in (3.7.4) and integrating from tl to t, we have t2 f (f (t,x (t),u(t,x (t), x (t,xo(t))) + Wt(tx (t)) tl n + Z xi (t,xo(t))fi (t, (t),u(t,x (t),c (t,xo(t)))] dt = 0, i=l where uo(t) = u(t,xo (t), x(t (t))) dx/dt = fi(tXo(t),uO(t)), i = 1,...,n, and hence

3.28 t2 n I[ o,U ] + [cot(t,xo(t)dt + c.i(t,xo(t))dx] = 0. t i=l The expression in brackets is now the differential of w taken along x (t). Hence I[Xo,Uo] + c(t2,x(t2)) - co(tl (tl)) = 0, I[XUO] = 1(tl, (37) since w(t2,x(t2)) = 0 and x(tl) = x1. By hypothesis (a) we have now for (t,x) E V and x = c (t,x), x n c (tx) + fo(t,x,u) + Z xi(tx)fi(t,x,u) > Ot(t,x) ~t O~ i=l X n + f(t,x,u(t,x,c (t,x)) + 7Z i (txU(t,)fi(t,x)) = 0, (3.7.6) where the second member is zero because of (b). By taking x = x(t), u = u(t) in (3.7.6) and integrating from tl to t3, we have t3 f (fo(t,x(t),u(t)) + cot(t,x(t)) tl n + W jxi(t(t))f,(t,x(t),u(t))}dt > O, (3.7.7) i=l where dxi = f.(t,x(t),u(t)), i = 1,...,n, and hence t3 n I[x,u] + f [ct(t,x(t))dt + E c i(t,x(t))dxi] > O. t1 i=l x

3.29 The expression in brackets is the differential of c taken along x(t). Hence I[x,u] + (t(tx(t)) - n(tl x(tl)) > 0) (3.7.8) I[x,u] > c(tl,xl), (3.7.9) since w(t,x(t)) = O, xl = x(tl) By comparison of (3.7.5) and (3.7.9), we obtain u(t 1x) = I[x0,uO] < I[x,u]. Remark. If V is the original domain of the given Lagrange problem, then statement (i) gives a global sufficiency condition. Remark. In order to obtain global sufficiency conditions, or at least, in order to enlarge as much as possible the domain V in E on which statement (i) is valid, the following remark is important. Assume that V can be divided into finitely many nonoverlapping subdomains V1,V2,..,VN by means of smooth n-dimensional manifolds so that (1) c(t,x) is continuous in V; (2) c(t,x) satisfies (3.7.9) in each of the subdomains V, s = 1,2,...,N. Then statement (i) still holds. We can even assume that V is the union of countably many subdomains Vj., j = 1,2,..., a.s above, under the additional hypothesis that any compact subset K in V intersects at most finitely many V (in other words, the subdomains V cluster at most on the boundary of V (or at infinity). 5

UNIVERSITY OF MICHIGAN 3II 9015 02656 6474