NECESSARY CONDITIONS FOR JOINING OPTIMAL SINGULAR AND NONSINGULAR SUBARCSt by J. P. McDanell*' and W. F. Powers** The University of Michigan Ann Arbor, Michigan April 1970 tThis research was supported by the National Science Foundation under Grant GK —4990 and the Natlonal Aeronautics and Space Administration under Grant NGR- 23 -005 3 29. *Graduate Student, Computer, Information, and Control Engineering Program. National Science Foundation Trainee. *'Assistant Professor, Department of Aerospace Engineering.

ABSTRACT Necessary conditions for the optimality of junctions between singular and nonsingular subarcs are developed for singular optimal control problems. Previously known necessary conditions concerning the continuity and smoothness of a piecewise analytic optimal control at a junction are clarified and extended. The main result is that the sum of the order of the singular arc and the lowest order time derivative of the control which is discontinuous at the junction must be an odd integer when the strengthened generalized Legendre-Clebsch condition is satisfied. Also, new necessary conditions which do not require an analyticity assumption are developed. These aid in characterizing problems which may possess nonanalytic junctions.

1. Introduction Optimal control problems in which the control variables appear only linearly admit the possibility of the occurrence of singular extremals. The analysis of such problems is complicated by the fact that the solution, in general, consists of some combination of singular and nonsingular subarcs, the number and sequence of which are not known a priori. If the solution is totally singular, recent results [1], [2] are available to prove optimality in a large number of cases. If the solution is totally nonsingular, it is the familiar bang-bang control generated by a switching function with isolated zeros, as determined by the minimum principle. However, the mathematical characterization of optimal controls which contain both singular and nonsingular subarcs is far from complete. This paper is concerned with the problem of characterizing the continuity and smoothness properties of the optimal control at a junction between singular and nonsingular subarcs. The analysis was motivated by the preliminary results obtained in this direction by Kelley, Kopp, and Moyer [3] and Johnson [4]. We shall comment on their results in a later section.

2. Problem Statement The class of problems to which this analysis applies is the following. Determine the scalar control u*(t), t E [to,tf] which minimizes the functional t J(u) = G(tf, x(tf)) + LO (t x) + L (t, x)u]dt (2. 1) to0 where the system equation is x = f (t,x) + fu(t,)u (2. 2) subject to the constraints Iu(t)| - K(t), te [to tf] (2.3) {to 1x(to), tfx(tf)} E S (2.4) Here x is an n-vector and S is a closed subset of R. The scalar functions f %, fu L0, LU are assumed to be analytic in both arguments in a suitable domain, and K(t) is assumed to be continuous and piecewise analytic for t E [to0 tf], i.e., K(t) is permitted to have a finite number of "corners". Of course, the usual case Jul - K with K = const. is included as a special case. We restrict attention to a scalar control in order to simplify notation. A similar analysis holds for each component of a vector control. Clearly, the Hamiltonian for this problem is linear in the control, i. e., H,x, Xu) -X fo(t,x) + Lo (t, x) + [ f (t, x)+ L (t, x)]u (2 5) The multiplier equations are then given by X = -H (tK,x,,u) (2.6) x where H is also linear in u. The coefficient of u in (2. 5) is called the switching function, which we shall designate as v (t), i. e., +(t) -H (t,x(t),X (t)) (2. 7) The mninimum principle (i. e., Pontryagin s maximum principle in a minimum form) states that for almost every t E [to,tf] and each u satisfying

lul< K(t), the optimal control u* (t) must satisfy H(t, x (t), X(t), u (t)) H (t, x (t), X (t), u) (2. 8) Therefore, as is well known, on each open subinterval of [to,tf] there are two distinct possibilities for u*. Either u,(t) = -K(t) sgn +(t) (2. 9) or (t) 0 (2. 10) Equations (2. 9) and (2. 10) define, respectively, the nonsingular and singular subarcs of the optimal control. The class of problems defined above will be called singular control problems, even though only a portion of the total trajectory may be singular.

3. Notation and Definitions The following definitions will clarify the terminology used in this paper. DEFINITION 1. A function g is said to be piecewise analytic on an interval (a,b) if it is piecewise continuous on (a,b) and analytic on each subinterval for which g is continuous. DEFINITION 2. A junction between singular and nonsingular subarcs of the control is said to be a nonanalytic junction if the control is not piecewise analytic in any neighborhood of the junction. DEFINITION 3. In a singular control problem, let (d q/dt2q)[H (t,x,X)] be the lowest order total derivative of H in which u appears explicitly. Then the integer q is called the order of the singular arc. Implicit in Definition 3 is the property that u first appears explicitly in an even order derivative of HU, i.e., it is correct to refer to q as an integer. For a proof of this property see Robbins [5]. We also need the well known generalized Legendre-Clebsch necessary condition for optimality of singular subarcs [3]. THEOREM. (Generalized Legendre-Clebsch condition) On an optimal singular subarc of order q, it is necessary that q a r d? 1 (-1)H 0 (3.1) Ldt- u Condition (3.1) hereafter will be called the GLC condition. By the strengthened GLC condition we mean that strict inequality holds in (3. 1). In this paper it will be convenient to consider the lowest order derivative of a function to be its zeroth derivative, by which we mean the function itself. We shall use the notation (0) (i) dg Also, where the context makes the meani]ng clear, we shall use u to designate the optimal control instead of u*.

4. The Junction Theorems As indicated in Section 1, the theory for totally singular and totally nonsingular optimal controls is rather well developed. The main difficulty with singular control problems occurs when both singular and nonsingular subarcs are present. Since a useful sufficient condition for such problems is not available, one is led naturally to the study of necessary conditions which are valid in the neighborhood of a junction between singular and nonsingular subarcs It is expected that such conditions can be used to eliminate candidate extremals and/or predict beforehand the way in which singular and nonsingular subarcs must be joined, e. g., whether the optimal control is continuous or discontinuous at a junction. If the optimal control is well-behaved in a neighborhood of a junction, then the following property must hold. THEOREM 1. Let t be a point at which singular and nonsingular subarcs of an optimal control u are joined, and let q be the order of the singular arc. Suppose the strengthened GLC condition is satisfied at tc, i. e., (-)q (8/ a u)H () c > 0, and assume that the control is piecewise analytic in a neighborhood of t Let u(r) (r _ 0) be the lowest order derivative of u which is discontinuous at t. Then q + r is an odd integer. Proof. Since H (q) is the lowest order time derivative of H which u u contains u explicitly, from (Z.2), (2. 5), and (2. 6) we see that it must have the form H ) (tx,, u) A(t,x, X) + B(t,x,X)u (4. 1) u Define the functions ar and D as follows. a(t) - A(t, x (t), X (t)) (4. 2) p(t) - B(t,x(t),X (t)) (4.3) From the hypotheses it is clear that a and P are continuous and have at least r continuous derivatives at t. The switching function ~ as defined by (2. 7) has exactly 2q +t r -1 continuous derivatives at tc.

Let E be a nonzero real number of arbitrarily small magnitude such that t + E is a point on the nonsingular side of t and t - E is a point on the C C C singular side. Let u and u designate the control u on the nonsingular and n (i)i) singular sides of t, respectively. By u(i) (t ) and u) (t ) we mean the limit C' n c s c as E - 0 of u( )(t + E) and u (tc - E), respectively. We wish to expand 4 (t + E) in a Taylor series about t. Let k = 2q + r. Then (k) will be the lowest order derivative of the switching function % which is discontinuous at tC, and since 0 on the singular side of t, the first nonzero term of the Taylor series will be the term containing Noting that d(k) K_ d [<x+ pu (4. 4) we can write. - (r) r r-i u C( =kt )+E) +S (t = -.(t) +o(E) (4. 5) C - E (rC i=y ir$l(tc) n c where Leibniz's formula [6] for differentiation of a product has been used to differentiate pun On the singular arc (aq) a= + pu 0 (4.6) Therefore, r — u, and S (r) = dr r-u (r (r-i)() (4. 7) dtr i Substituting from (4. 7) into (4. 5) k k +(t+) k! -i) c n S(i)(tc +0 (E) (4. 8) If r> 0, u (tc) = U )(tc), i = 0,.. r - (49) Therefore, (4.) becomes Therefore, (4.8) becomes

(t+E k! (t)[U(r) (t - u(r)(t )] + (Ek (4. 10) Let C = -sgn 4(tc + E) so that u (t) = -K(t). Then recalling that u(i) (tc) lurn (i) cn n c =E u() (t + E) we have n c u n(tc )a- K (tC) i = O,. r (4. 11) Now consider the following series expansion on the singular arc. aK~t - E) - (t- E )K- (i) (i) ) - Ui (t) The right hand side of (4. 12) can be simplified using (4. 9) and (4. 11) to obtain K(t -E)-(t -E)= (t) (t + (E) (4. 13) Substituting from (4. 13) into (4. 10) and recalling that k = 2q + r, we obtain (tc +E) = (-1)r k.! 1(t)[K(c Et)(tc )] + O(E) (4. 14) From the application of the minimum principle on the nonsingular subarc (Equation (2. 9)) we have - = 1 if 4(tc + E) < 0 and c = -1 if 4(tc + E) > 0. Therefore the following inequality must hold. r zq (-1) (tC)[K(tC- E)+ U(tc- E)] < 0 (4. 15) From the GLC condition we have (-1)q P(t) > 0 (4.16) Multiplying the left hand side of (4. 15) by the positive quantity in (4. 16) we obtain (-1)+r q Z(t )[K - E) u(t - E)]< 0 (4. 17) Since Ju(t)I _- K(t) for all t E [to,tf], and the singular arc is tacitly assumed to be interior almost everywhere, the bracketed quantity in (4. 17) is strictly

positive, regardless of the choice of sign on +u(t - E). Also e > 0 regardC less of the sign of E. Therefore, condition (4.17) reduces to (-1)q+r < 0 (4. 18) from which it is clear that q + r is an odd integer. This completes the proof. Theorem 1 implies the following important corollaries. COROLLARY 1. In q-even problems, assuming u is piecewise analytic and the strengthened GLC condition is satisfied, the optimal control is continuous at each junction. COROLLARY 2. In q-odd problems, assuming u is piecewise analytic and the strengthened GLC condition is satisfied, the optimal control either has a jump discontinuity at each junction, or else the singular control joins the boundary smoothly, i. e., with continuous first derivative. In the corollaries above, especially Corollary 1, which applies to the q-even case, the assumption that u is piecewise analytic is not to be taken lightly. In fact, the authors have not seen or been able to produce a q-even example with a continuous junction, i. e., the junction is usually nonanalytic if q is even. The conclusions reachedbyKelley, Kopp, and Moyer [3] are consistent with those stated in Corollaries 1 and 2, with one important exception- they ruled out the possibility of a continuous junction for q- odd problems. This erroneous conclusion resulted from the claim that continuity of u implies (a/lau)H(2q) > 0 which is not true in general, as can be seen from (4.15) (in u (z-q) which P = (8/8u)Hq). That such a junction is realizable will be demonstrated u by means of a simple example in a later section. Theorem 1 requires that the strengthened GLC condition be satisfied at the junction point. While this is the usual and most important case, the possibilityg exists that the GLC conditionr is satisfied with equality To Ttreat this case, note from Definition 3 that for a qth order singular arc the GLC expression (8/5u)H(2q) (i.e., p) cannot be identically zero on lthe singular subarc. Therefore, in view of our analyticity assumptiions, a derivative of some

order must be nonzero at the junction point tC even if p (t ) = 0. This leads to the following theorem, which is a generalization of Theorem 1, but is stated separately to avoid obscurirng the result for the important case covered by Theorem 1. THEOREM 2. Let t be a point at which singular and nonsingular subarcs of an optimal control u are joined, and let q be the order of the singular arc. Assume that the control is piecewise analytic in a neighborhood of tc. Let u(r) (r -' 0) be the lowest order derivative of u which is discontinuous at t, and let p (m) (m - 0) be the lowest order derivative of the GLC expression (a/au)H(zq) p which is nonzero at t. Then, (i) if m' r, q + r + m is an u~(m) + m) q+r+m odd integer; (ii) if m > r, -sgn[p() (tc) () ( = Proof Outline. The proof is similar to that for Theorem 1; however, in order to obtain a nontrivial term in the Taylor series expansion for (tc+ +), one must consider higher order terms with the result that (4. 15) is replaced by ( -1) e Mq ((n )(ic) [K (tc E)~us (t - E)] < 0 (4. 19) A Taylor series expansion for P(t) on the singular arc yields P(t e) = () (t ) + 0 (m).(4. 20) where the subscripts s and n on. (m) (t ) indicate the limit at t on the singular and nonsingular sides, respectively. Since P(m )(tc) 0, from the GLC condition and (4. 20) we have ()q+mm (m) (t ) > 0 (4. 21) From (4. 19) and (4. 21) it follows that (1)q+r+m P(m) + (m) (t < 0 (4. 22) c C If m' r, pfm) is continuous at tc, in which case (A. 22) implies that q+r+m is an odd integer. If m> r, p(m) may not be continuous at tc, and the conclusion of Theorem 2 for this case follows,

10 The main restriction in Theorems 1 and 2 is the assumption that the control is piecewise analytic in a neighborhood of the junction. This hypothesis is usually satisfied on the singular subarc, but not always on the nonsingular subarc. Thus, we are led to consider properties which do not require the assumption of analyticity, as stated in the following theorem. The functions A and B in this theorem are those defined by the identity (4. 1). THEOREM 3. Let u be an optimal control which contains both singular and nonsingular subarcs, where the singular subarcs are of order q. Then, (i) if H q) O on the nonsingular side of a junction, the control must be discontinuous; (ii) if H(zq) 0 on the nonsingular side of a junction and B # 0 at the junction, the control must be continuous; (iii) if A 0 and K(tc) # 0 at a junction point tc, the control must be discontinuous. Proof. Using the same notation as in the proof of Theorem 1, and re+ (t )u (4.23) calling that H(zq) 0 on the singular subarc, we have for case (i) (t ) + P (t )tc un (tc) + ((C) c s c from which we obtain u (tc) us (tc). For case (ii) we have at ) + P(tc)un ) = o = a(t) + (tc)us (c) (4. 24) c c n ( tc c ( c Since P(tc) 0, we must have u (t ) = u (t ). For case (iii) A= 0 implies us 0, and since lun(tc)= K(t) 0, the control must be discontinuous. Case (iii) of Theorem 3 may appear to be a rather special case, but it occurs frequently enough to be of interest Note that this result is independent of an even or odd assumption. Because of this, we can couple (iii) with the previous result for q-even problems to obtain the following interesting property. COROLLARY 3. If q is even, A -0, K(t ) # 0, and P(t ) =B(tc,xtC), X ~tk)) # 0, where t is a junction point between optimal singular and nonsingular ~c) 0whr!c

subarcs, then the junction is nonanalytic. Proof. Assume the contrary, i. e., that the optimal control is piecewise analytic in a neighborhood of tc. Then by Corollary 1, the control is c continuous at t. But, by Theorem 3 (iii) the control is discontinuous, which supplies the necessary contradiction. In the next section this corollary will be used to predict the nonanalytic junction in the well-known Fuller problem.

12 5. Example of a Nonanalytic Junction Consider the Fuller problem [7 ], which is to minimize T J -x Sxidt (5.1) subject to xl = x, xl (O) = X1? O (5. 2) z = u, x2 () = z [uj- 1 (5.3) where T is fixed. The Hamiltonian, the multiplier equations, and the switching function are given by H = X x2 + X2 u + -x (5. 4) 1 = -x1, X (T) = (5.5) 2z = -X1, X2(T) =0 Hu Xz (5.6) The lowest order derivative of H which contains u explicitly is H(4) u (5.7) u from which we see that the order of the singular arc is even, namely q = 2 Also, the strengthened GLC condition holds, and A(t,x,X) -0. Thus, we have precisely the conditions of Corollary 3, indicating that any junctions which occur must be nonanalytic junctions. This problem has been studied thoroughly by Fuller [7] and Johansen [8], and the result is well known. The singular arc is given by u = = x= (5.8) s Since! f O0, the initial control must be nonsingular. The nonsingular arc is characterized by the nonlinear differential equation (4) = -sgnd~ (5.9)

The solution of (5. 9) yields a switching function with an infinite number of zeros such that the ratio of the lengths of successive intervals between zeros is a constant. If T is sufficiently large, the resulting nonsingular (bang-bang) control drives the state to the origin in a finite time t, with an infinite number of switches occurring in a neighborhood of tc, at which point the optimal control becomes singular. The control is clearly discontinuous at the junction point tc as it must be according to (iii) of Theorem 3. Even though q is even, Corollary 1 is not violated because the control is not piecewise analytic in a neighborhood of the junction. The predicted behavior at the junction is useful knowledge for numerical computational schemes, e. g., Jacobson [9] was able to successfully compute bang-bang solutions for this problem with T sufficiently small so that the singular arc did not occur. However, for large T the nonanalytic junction came into play. After computing about ten switches, the method became unstable [10]. Note that the optimal control for this "innocent looking", second-order example is measurable but not piecewise continuous. Aside from its physical applicability, the existence of such examples is useful for motivating the assumption of measurable controls in the proof of the minimum principle.

14 6. Example of a Smooth Junction with q Odd This example demonstrates not only the realizability of a smooth junction with q odd, but also the dependence of junction phenomena upon boundary conditions. For this case we consider the performance index T J = -S~xz - Xl)dt (6.1) 0 where T = 2. 985. The equations of motion and constraints are given by x1 = x, x (0) = 0, x1 (T) = r (6.2) x2 = u, x2(0) = 1, xz(T) = cz lull 1 (6.3) The Hamiltonian and the multiplier equations are given by H = XXz2 + z2 U + -xz -- X12 (6. 4) A1 = X1 (6.5),2 = -X1 - X2 The switching function and its second derivative are AH -Hu = X2 (6.6) H = -xl - u (6.7) so the singular arc is first order, i. e., q is odd. From (6.7) we see that the strengthened GLC condition is satisfied. Setting the right hand side of (6. 7) equal to zero and substituting in the equations of motion (6. 2), it is readily verified that a singular arc emanating from the initial state (0, 1) is given by u = sin t xi = sin t (6 8) 2 = COS t

15 If the terminal state (lo-,ar2) is chosen to lie on the trajectory (6. 8), the solution is totally singular, as can be shown by the sufficient conditions in References [1] and [2]. However, in this paper we are concerned with junctions. Consider the case where orl = 0, crz = -G 7. For this case we propose as a candidate for the optimal control -sint, tE [0,2) u { (6.9) -1, tE [TT] This control is admissible and satisfies all the necessary conditions for optimality, including that of Theorem 1. There is a junction at t =. The conc 2. trol and its first derivative are continuous at t, but the second derivative is discontinuous, so we have r = 2, q = 1, and q + r is an odd integer. The authors are unaware of any workable sufficient conditions in the literature which are applicable to this particular type of problem, i. e., nonconvex and containing both singular and nonsingular subarcs. Consequently, we employed a gradient type numerical method to justify that the candidate control is indeed optimal, within the bounds of a numerical justification. The modified conjugate gradient method of Pagurek and Woodside [11 ] was used with penalty functions to enforce the terminal constraints. The result is shown in Figure 1. The control is continuous and smooth at the junction as expected. It is apparent that the fortuitous occurrence of this smooth junction is a direct result of our judicious choice of the terminal boundary conditions. In fact, to generate this phenomenon, the form of the candidate control was first selected on the basis of intuition; then a convenient poiht on the resulting trajectory was selected as the fixed terminal state, and finally the corresponding time was taken to be the explicit final time. By changing the terminal state, we were able to generate discontinuous controls, which undoubtedly are the usual case. These are shown in Figures 2 and 3. For these cases r - 0, and the condition of Theorem 1 is satisfied again. To further emphasize the special character of the smooth junction, the phase plane trajectories for the controls in Figures 1-2 are given in Figure 4.

16 7. Conclusion Necessary conditions for the optimality of junctions between singular and nonsingular subarcs in singular optimal control problems have been developed. Necessary conditions developed previously by Kelley, Kopp, and Moyer [3], which involve an analyticity assumption, have been clarified and extended. The main result in this direction is that the sum of the order of the singular arc and the order of the lowest time derivative of the control which is discontinuous at the junction must be an odd integer when the strengthened generalized Legendre-Clebsch condition is satisfied. Also, new necessary conditions which do not involve an analyticity assumption have been developed. These conditions aid mainly in characterizing problems which may possess nonanalytic junctions. It should be emphasized that these are local necessary conditions for optimality. Yet, as indicated by the example in Section 6, the point at which a junction occurs is determined mainly by initial and terminal boundary conditions, i. e., by essentially nonlocal information. This means that any junc tion theory which, for example, might be used to establish criteria for switching between singular and nonsingular arcs in an indirect computational scheme will have to take such nonlocal information into account. It is becoming increasingly apparent that a close relationship exists between singular problems and bounded state problems [12][13]. In this regard it is interesting to note that the result of Theorem 1 bears some similarity to a result of Jacobson, Lele, and Speyer [13] which identifies certain propertlies of optimal trajectories associated with odd order state space constraints, Such similarities suggest the possibility of a duality between these two classes of problems which might be profitably exploited.

RE FERENCES [1] J. P. McDanell and W. F. Powers, "New Jacobi-Type Necessary and Sufficient Conditions for Singular Optimization Problems," to appear in AIAA Journal. [2] J. L. Speyer and D. H. Jacobson, "Necessary and Sufficient Conditions for Optimality for Singular Control Problems: A Transformation Approach, " Report No. 69-24, Analytical Mechanics Associates, Inc., Cambridge, Mass., November, 1969. [3] H J. Kelley, R. E. Kopp, and H. G. Moyer, "Singular Extremals," in Topics in Optimization (G. Leitmann, ed.), Academic Press, New York, 1967. [4] C. D. Johnson, "Singular Solutions in Problems of Optimal Control," in Advances in Control Systems, Vol. 2 (C T. Leondes, ed.), Academic Press, New York, 1965. [5] H. M. Robbins, "A Generalized Legendre-Clebsch Condition for the Singular Cases of Optimal Control," IBM Journal, Vol. 11, 1967, pp. 361-372. [6] J. Dieudonne, Foundations of Modern Analysis, Academic Press, New York, 1960, p. 184. [7] A. T. Fuller, "Study of an Optimum Nonlinear Control System," Journal of Electronics and Control, Vol. 15,No. 1,pp. 63-71. [8] D. E. Johansen, "Solution of a Linear Mean Square Estimation Problem when Process Statistics are Undefined," Joint Automatic Control Conference, Troy, New York, June 1965. [9] D. H. Jacobson, "Differential Dynamic Programming Methods for Solving Bang- Bang Control Problems," IEEE Transactions on Automatic Control, Vol.. AC-13,No.6, 1968, pp. 661-675. [10] D. H. Jacobson, personal communication. [11] B. Pagurek and C. M. Woodside, "The Conjugate Gradient Method for Optimal Control Problems with Bounded Control Variables," Automatic a Vol. 4, 1968, pp. 337-349. [12] D. H. Jacobson, M. M. Lele, "A Transformation Technique for Optimal Control Problems with a State Variable Inequality Constraint,"' Harvard Technical Report No. 574, October 1968. [13] D.H. Jacobson, M. M. Lele, and JO L. Speyer, "New Necessary Conditions of Optimality for Control Problems with State Variable Inequality Con-s straints," Harvard Technical Report No, 597, August 1969.

u(t) -1.0 -,8 -.6 xl (T) = 0 X2 (T) - -| - -- Computed Control. l yProposed Optimal Control Tr T Figure 1. Example of an optimal control which is continuous at a singularto-nonsingular junction with q odd.

u (t) -.8 8-.8~. -.6/ / (T) =-.157 / x(T) = -1.492 -.4 / Computed Control -.2 / Proposed Optimal Control Ir T 4 Figure 2. A typical discontinuous junction.

u () X2(T) -1. 336 lo 3 Figure 3. A predominantly singular control with discontinuous junction to a short nonsingular subarc.

Z p.~e T s9anflTd u! SiOa~uoo an} ao sa Ioj0ao oa a} ueId aSetd *i a':S7n /~~~~~ /.' / / i Y/ if I() U~~~~~~~~~~~~~~~~~. // IXI'1A'+ -- 1-i