NOTE: Errata - follows page 336 TECHNICAL REPORT ECOM 01870-4-T July 1966 ON THE BEST CHEBYSHEV APPROXIMATION OF AN IMPULSE RESPONSE FUNCTION AT A FINITE SET OF EQUALLY-SPACED POINTS Technical Report No. 172 CONTRACT^NO. DA-28- 043-AMC-01870(E) DA Proieject Noo. 280-Mt-00287(E). repared By -. iRobert Fisch COOLEY ELECTRONICS LABORATORY Department of Electrical Engineering The University of Michigan Ann Arbor, Michigan for U. S. ARMY ELECTRONICS COMMAND FORT MONMOUTH, NEW JERSEY THE UNiR I Qr ENGINEERING LIBRARY

1, - ^I I ^.

ACKNOWLEDGEMENTS The author is deeply indebted to Professor Keki B. Irani, the chairman of his doctoral committee, who not only aroused his interest in the problem of Chebyshev approximations in the time domain, but who also was a source of constant encouragement and of valuable suggestions during the course of the research. He also wishes to thank the other members of his doctoral committee for their helpful comments concerning the material. Gratitude is also expressed to Professor B. F. Barton and colleagues, both past and present, at the Cooley Electronics Laboratory who have contributed immeasurably to the author's education. Finally, the author wishes to thank Miss Ann M. Rentschler, Mrs. Lillian M. Thurston, Mrs. Elizabeth W. Willsher, and Mr. Thomas E. Jennett for their extensive efforts in typing and preparing this manuscript for publication. In addition, he wishes to thank his wife, Ruth, for her patience and help in proofreading the final draft. A part of the research reported in this dissertation was supported by the U. S. Army Signal Corps.

TABLE OF CONTENTS. Page ACKNOWLEDGEMENTS LIST OF TABLES vi LIST OF ILLUSTRATIONS vii LIST OF SYMBOLS viii LIST OF APPENDICES Xxiii ABSTRACT xxiv CHAPTER I: INTRODUCTION 1 1. 1 Notation and Definitions 2 1.2 Selection of the Approximating Function, Exponential Representation 7 1.3 Selection of the Criterion of Approximation 12 1. 4 Formulation of the Problem 13 1. 5 Statement of the Problem 22 1.6 Plan of the Thesis 24 CHAPTER II: STATE-OF-THE-ART 26 2. 1 Introduction 26 2. 2 The L -Approximation Problem of Network Synthesis 34 2.2. 1 L -Approximations 34 2 2.2 L,-Approximations 37 2.3 The q -Approximation Problem of Network Synthesis 37 2.3. 1 The Approximation Problem when the Matrix [E] is Initially Prescribed 40 2.3. 1. 1 The Least-Squares Approximation Problem 41 2.3.1.2 The Chebyshev Approximation Problem 42 2. 3. 2 The Original Method of Prony 64 2. 3. 3 The Approximation Problem when the Matrix [E] is Not Initially Prescribed 76 iii

TABLE OF CONTENTS (Cont.) Page 2.3. 3. 1 The Least-Square Approximation Method of Yengst 79 2.3.3.2 The Chebyshev Approximation Method of Ruston 83 CHAPTER III: PRONY'S EXTENDED METHOD 89 CHAPTER IV: OPTIMAL CHEBYSHEV APPROXIMATIONS AT DISCRETE POINTS IN THE TIME DOMAIN 106 4. 1 Introduction 106 4. 2 General Approach to the Solution of this Chebyshev Approximation Problem 107 4. 3 The Existence of the Best Chebyshev Approximation 109 4. 4 Properties of the Best Chebyshev Approximation 131 4. 5 The Best Chebyshev Approximation in a (2n+1)-Dimensional Reference Subspace 142 4. 6 On the Uniqueness of the Best Chebyshev Approximation 166 4.7 Summary 172 CHAPTER V: COMPUTATIONAL METHOD AND EXAMPLES 173 5.1 Introduction 173 5. 2 The Method of Descent 173 5.3 The Computational Procedure 176 5. 3. 1 The Algorithm 180 5. 3. 2 Comments on the Algorithm 186 5. 4 Numerical Examples 187 5. 5 Analysis of Examples 244 CHAPTER VI: THE DISCRETE TIME DOMAIN CHEBYSHEV APPROXIMATION PROBLEM OF NETWORK SYNTHESIS 248 6. 1 Introduction 248 6. 2 Determination of the Vector Pair (a,,s) from the Vector Pair (Sz). 249 6.3 Physical Realizability 253 6. 4 Selection of the Finite Approximation Point Set, T 256 iv

TABLE OF CONTENTS (Conto) Page 6. 5 Choice of the Dimensions of the Parameter Spaces 259 6.6 Synthesis Procedure 259 6.6. 1 A Simple Example of the Synthesis Procedure- 261 CHAPTER VII: CONCLUSIONS AND AREAS OF FUTURE WORK 267 7.1 Discussion of Results 267 7.2 Limitation of the Results 270 7.3 Future Areas of Study 271 REFERENCES 329

LIST OF TABLES Table Title Page 4. 1 Results of Example 4. 2. 130 5.1 Results of Step 19 of Example 5. 1. 197 5. 2 Results of Step 28 of Example 5. 1. 200 5.3 Results of Step 45 of Example 5. 1. 203 5.4 Results of Step 2 of Example 5. 2. 208 5. 5 Values of E.(r) for Steps 3 and 10 of Example 5.2. 210 5. 6 Results of Steps (14) through (16) of Example 5.2. 214 5. 7 Results of Steps 21 and 22 of Example 5. 2. 218 5. 8 Results of Steps 3 and 6 of Example 5. 3. 222 5. 9 Results of Steps 11 and 13 of Example 5. 3. 226 5. 10 Results of Steps 4 and 5 of Example 5. 4. 231 50 11 Results of Steps 6 and 7 of Example 5. 4. 233 5. 12 The results of the minimization procedure. 237 5. 13 The best Chebyshev parameter vectors of Example 5.6. 242 5. 14 Comparison between the initial and final values of IIIoo. 244 vi

LIST OF ILLUSTRATIONS Figure Title Page 1 The best approximation to f onto Vn in Eq forp = 1, 2, oo, where q = 2, n = 1. 29 2 Geometric interpretation of Lemrma 2. 1. 52 3 Geometric interpretation of Theorem 2. 3. 57 3 4 Approximation of f in E onto C1(E) in the Chebyshev sense. 63 5 The points in the 9z x space considered in Lemma 4. 1. 112 6 The Chebyshev approximation error of Example 5. 1. 205 7 lIe(~r)ll vs. r for Example 5. 1. 206 8 Chebyshev approximation error of Example 5.2. 220 9 Example of a prescribed irpulse response function h(t). 257 10 Impulse response function, h(t) = 262 (l+t)n, ~11 Approximation error function, E(t) = h(t)- h**(t). 265 12 Network realizing h**(t) as an input-tooutput voltage ratio. 266 vii

LIST OF SYMBOLS General Conventions 1. Lower case letters are used for constants, scalar-valued variables and functions, e. g, a, p, f(t). 2. Underlines lower case letters denote-vectors, e. g., f, s, z, a, A. 3. Square-bracketed capital letters denote matrices, e. g., [A], [E(s)], [Z(z)], [Z (z)] 4. Capital script letters denote sets, e.g., s',' 9Z z. 5. g( ) or g(, -) denotes a function, with the dot standing for an undesignated variable. 6. Braces denote a set or a family, e.g., {x1, x2,..., xn} is the ordered set representing the components of x. {x: P} is a set of x's having property P. 7. (, s) denotes the ordered pair of vectors a and s. 8. A bar (-) over a symbol denotes the complex conjugate value. 9. A starred symbol denotes the optimum value, e. g., f*(t), f**, r*. Specifically, a single star denotes the optimum value with respect to one set of parameters and a double star denotes the optimum value with respect to two sets of parameters. 10. Superscript minus one ( 1) denotes the inverse of a matrix, e. g., [E] 1 11. Superscript dagger (+) denotes the pseudo inverse of a matrix, e.g., [E]+. 12. Superscript tilde (~) denotes the estimate of the parameter vector, e.g., r, z. viii

LIST OF SYMBOLS (Cont. ) 13. Superscript hat ( ) denotes the approximating function or vector, e.g., f(t;a, s), f(, s). T 14. Superscript T ( ) denotes the transpose of a matrix, e. g., [A], [R] 15. Vectors with letter superscripts denote the projections of the vectors onto a lower dimensional reference subspace having the same fixed basis, e.g., f(k), (k) 16. A matrix with a letter superscript denotes (a) submatrix; or (b) the form of the matrix in a lower dimensional reference subspace, e.g., [E(k)], [Z(k)(z)], [A(k)]. General Symbols and Abbreviations A ~= ~ equals by definition; denotes e is an element of I' ~ does not belong to U union (0 direct sum x Cartesian product (-, ~ ) the inner product relation II- II the p-th norm, where p > 1 p x: P} the set of x's having property P (a, b) open interval a < t < b [a, b) semi-closed interval a < t < b ix

LIST OF SYMBOLS (Cont.) [a, b] closed interval a < t < b C (E) the n-dimensional column space of the matrix [E] Re{ s} the real part of s max maximum over i max{a,b} max{a,b} = a ifa> b, max{a,b} = b if b> a sgn signum function: sgn x = ~iT for all x O sgn 0 = 0 Symbols with Special Meaning Defined or first Symbol Description used on page a an arbitrary real constant 30 a the real constant defined by Eq. 2.36 50 [A] a qxn real matrix 40 CXS ~ the set of all parameter vectors a s (Definition 1.6) 10 Xd x P the set of the parameter vector pairs s (,s) 11 b an arbitrary real constant 30 9ffl, ~ the set of all parameter vectors for a fixed z e. (Definition 1.8) 21 c an arbitrary real constant 50 c as a subscript or superscript denotes the index v which yields the best Chebyshev approximation in Uq 60 x

LIST OF SYMBOLS (Cont.) Defined or first Symbol Description used on page C (E), C (Z) the n-dimensional column space of the matrix [E] (or [Z]) 53 d1, d2 arbitrary constants 114 D.(r) a polynomial in the components of r E? 148 skt e the k-th exponential function 8 ek the q-dimensional vector repreSk1 senting the ordered set {e, kt2 ski2 s kq e,e e,k } of the s t exponential function e. 14 e.i the elements of the matrix [E] 1 ~ defined by Eq. 1. 12 15 [E(s)], [E] a qxn matrix whose column space defines Vn(s) (see Eq. 1. 12 or Eq. 1. 12a) 15 (k) [E(k)] an (nx 1)xn matrix submatrix of [E] given by Eq. 2.32 49 Eq the real q-dimensional vector space 4 the set of all vector E(r) E U which satisfy Eq. 3.16 99 f(t) the prescribed function of t 30 f the prescribed real q-dimensional vector 27 (3,z), f(a,s) the real approximating vector in Uq 89 xi

LIST OF SYMBOLS (Cont. ) Defined or first Symbol Description used on page f, f*, the best real approximating vector ^ in Uq 101 f(,k** z*) f(k) f (k) the projections of the vectors in Uq and U 48 f(W) (W) the projection of the vectors in Uq onto the reference subspaces U2n+l 133 w f (k)? Zk) the best Chebyshev real approxima^- ~~k' k ting vector in Un+134 [F] a n x(n+l) real matrix defined by Eq. 2.71 75 [F] the (q-n) x (n+l) real prescribed matrix defined by Eq. 4.34 120 a q-dimensional real vector in Uq 28 g the q-dimensional real vector in C- (R) 92 q- n h(t) the prescribed impulse response function 2 h(t;a, s) the approximating impulse response function. (Eq. 1.3, Eq. 1.3a) 8 h the prescribed q-dimensional real vector —the ordered set of values {h(t), h(t2)..., h(t)} 14 q h, h the (q-n)-dimensional prescribed real vector defined by Eq. 2.80 (or Eq. 2. 84) 79 h(a, s) the q-dimensional real approximating vector 14 xii

LIST OF SYMBOLS (Cont.) Defined or first Symbol Description used on page h(t; a *s), the best Lp-approximating function P - of h(t) with respect to a e cIg for h *(t;s), a fixed s e 97. (Definition 1. 1) 5 p h*(t; s), h*(t) hp*(t) h(t; a** s*), the best L -approximating function h(tp *) with respect to (a, s) c Js. x 99 (Definition 1.2) 5 h**(t) the best Chebyshev approximating function at a finite equally- space discrete value of t with respect to (a, s)e x -92 248 h*(s), h(a*, s) the best _q-approximating vector h(s), ( ) real vector with respect to a E. h*(s), h~;* s) S p p'- for a fixed s E. (see Definition 1.3) 6 c(a**, s*), the best I -approximating real vec- p tor with respect to (a, s) e "? x,9 h** h** (Definition 1. 4) 6 — p - 5 [H] the (q-n)x(n+1) real matrix defined by Eq. 2.83 81 [H ], [Ho] the (q-n) x n matrix defined by Eq. 2.80 (or Eq. 2.84) 79 jr the set of approximating functions {h(t;ta, s):t E, ac e s' s e 9} 2 i the index{i=l,..., q} or by Eq. 2.25 47 xiii

LIST OF SYMBOLS (Cont.) Defined or first Symbol Descriptio used on page j the order of the repeated root 8 k a set of n indices,{k= 1, 2,..., n} 8 k an arbitrary index in the set of (m) indices 47 Iq I-q ~ Banach space of q-tuples 5 p q-approximation the approximation which is measured by the P1-norm of the error vector. 5 t -projection the best mapping of the prescribed -P vector in q- space onto the approxip mating subspace Vn 27 L -space Banach sapce of measurable function Lp()spae on interval 3 L space L -approximation the approximation which is measured by the L -norm of the error function,-e (t):' 3 L -projection the best mapping of a point in an Lp-space onto the approximation subspace 27 m the dimension of the reference subspace 47 m the number of components of the error vector whose absolute value are equal, m > n+1 146 m arbitrary dimension of the vector space 50 xiv

LIST OF SYMBOLS (Cont. ) Defined or first Symbol Description used on page n the dimension of the parameter spaces 9'ss S, 9z' 9 N.(r) a polynomial in the components of ^J r e1 148 p a constant > 1 used in related problems 3 P (z) n-th order polynomial in z whose leading coefficient, rn= 1. See Eq. 2.60 66 P(z) the general n-th order polynomial equation defined by Eq. 2.65 or Eq. 3.4 72 P'(z) the first derivative of the polynomial P(z) 95 P"(z) the second derivative of the polynomial P(z) 95 q the dimension of the prescribed vector space, i. e., the number of discrete values of t 4 r the (n+1)-dimensional parameter vector, defined by Eq. 2.66 (or Eq. 3. 3), denoting the coefficients of the polynomial P(z) 72 r the vector r e I' which yields the minimum value of 11 [F] r II in the set 11r 11 = 122 r' an n-dimensional vector defined by Eq. 2.77 78 xv

LIST OF SYMBOLS (Cont ) Defined or first Symbol Description used on page r' an arbitrary (n+l)-dimensional vector r in 123 r' the best estimate of the n-dimensional vector r' obtained by Ruston (or Yengst) 79 r(j) the vector r e 4 whose j-th component is equal to one 128 r* the best Chebyshev solution vector in - (see Definition 4. 1) 120 r * the best Chebyshev k-th reference soKk lution vector in R (see Definition 4. 2) 139 r the best k-th Chebyshev reference solution vector in 1 when all the components of (2n+1 nents of e( *(r) e Uk2 are equal in absolute value to ll *(k)(r)ll (see Definition 4.3) 151 ro the best Chebyshev k-th reference solution vector in A? when 2n components *(k) 2n+1 of E*(r) e Uk are equal in absolute value. (see Definition 4. 4) 158 [R(r)], [R] is a qx(q-n) matrix defined by Eq. 3.2, which is a function of r. If q= 2n, then [R] is defined by Eq. 2. 69 90 ^,~~ -the set of all parameter vectors r (see Definition 3.1) 91 s the n-dimensional parameter vector denoting the n-exponents (Eq. 1. 5) 9 -,~- the set of all parameter vectors s (Definition 1.5) 10 xvi

LIST OF SYMBOLS (Cont.) Defined or first Symbol Description used on page t time 2 At time interval (ti- ti), i= 1, 2,...., q 22 ~'the initial sample time 4 tI the final sample time 4 q T as a superscript: indicates transposition 42 the finite point set {t.:i=1,2,..., q} 4 T the finite equally-spaced point set e {r t +(i-1) At: i=l, 2,..., q 22 [T] qx q diagonal matrix whose elements are {tl,...,t 16 the semi-infinite interval [0,c) 2 Uq the complex q-dimensional vector space 4 U the m-dimensional k-th reference subspace of Uq (Definition 2. 1) 47 Un+ the set of (n+l)-dimensional reference v subspaces, where v= 1,2,..., (n ) 48 U the set of (2n+ l)-dimensional reference subspaces, where w=1, 2,...,(2 1) 132 v the set of indices, v= 1, 2,..., ( 1) 48 v(z) a vector representing the orthogonal projection of f onto C (Z) 103 Vn, Vn(s) the n-dimensional approximating subspace 15 xvii

LIST OF SYMBOLS (Cont. ) Defined or first Symbol Description used on page q w the set of indices {1, 2,.., (2n+)} 107 w(r) a vector representing the orthogonal projection of f onto Cq n(R) 103 q- n x an arbitrary vector in Em 50. an arbitrary vector in Em 50 z the n-dimensional parameter vector, defined by Eq. 1.19. The component skAt Zk= e 18 z the estimate of the vector z e 5 obtained by Ruston (or Yengst) 80 [Z(z)], [Z] the qx n matrix which is a function of the vector z E 5 and defined by Eq. 1. 18 (or Eq.. 18a, 18c) 18 [ZW(z)] is a (2n+l)xn submatrix of [Z(z)], see Eq. 4.45 133 5$^; the set of all parameter vectors z Definition 1.7 21 a the n-dimensional parameter vector denoting the n-coefficients of the nexponential functions (Eq. 1. 4) 9 _a k, a * the vector a e's defining the best Chebyshev approximating vector in Uk 53 (a, s) the ordered vector pair of the parameter vectors a and s 2 xviii

LIST OF SYMBOLS (Cont ) Defined or first Symbol Description used on page (a*, s), (a*, s) the best L p(2)-approximating parameter vector a e iS for a fixed s e 5 (a* s*), (a**, s*) the best Lp(.q )-approximating parameter vector pair in ts x 97 5 3 the n-dimensional parameter vector defined by Eq. 1. 22 21 (_f, z) the ordered parameter vector pair,3 and z 20 (3**, z*) the best Chebyshev vector pair in gz~x< 106 (*, zk*) the best Chebyshev k-th reference parameter vector pair in z x a 134 y an arbitrary vector in Eq n 92 6(r), 6 is a (q-n)-dimensional unknown real vector defined by Eq. 2.75 (or Eq. 2.76) 77 6, 61' 62 an arbitrary positive constant 111 (t), e (t; a, s) the approximating error function 3 E*(t), E*(t;s), the error function resulting from the best L -approximation with e (t;c* s) respect to a e sg for a fixed s E P -Pt (Definition 1.1) 5 **(t) the error function resulting from the best Lp- approximation with respect to (, s)e,S' x 9P (Definition 1.2' 5 xix

LIST OF SYMBOLS (Cont. ) Defined or first Symbol Description used on page E, E(.), e(,) the q-dimensional real error vector, with the dots standing for the undesignated parameter vector 4 e*( ), e*( ) the real error vector resulting from the best Ii-approximation with respect to gne parameter 6 E *(r), e*(r) the best real error vector in Uq which satisfies Eq. 3. 16 for a fixed r e 4 99 * E** the real error-vector resulting from the best fq-approximation with respect to (a, s)'E sd x 9P (Definition 1. 4) 6 *(r*) the error vector E** 101 e(k), (k)( (k)( the projection of the vectors in Uq onto the k-th reference subspace Uk, with the dots standing for the undesignated parameter vectors 53 e*()(r) the best Chebyshev error vector in U2n+ for a fixed r 146 k E*(k)(rk*), (k(*r) **(k) the best Chebyshev error vector in Uk (see Definition 4. 2) 139 a real constant 166 7?1 ~ an arbitrary positive constant 111 71 the minimum value of Il [F]3r II defined byEq. 4.38 c 121 xx

LIST OF SYMBOLS (Cont. ) Defined or first Symbol Description used on page k(k) a real n+l dimensional vector, defined by Eq. 2. 42, which is orthogonal to the n-dimensional approximating subspace in Un+ 54 X(W)(r) a nonzero element of the matrix 121(W) [A(W)(r)] 138 (k) X k(r) the j-th column vector of the matrix [A(k)(r) 146 [A(W(r)] a (2n+1) x (n+1) matrix, defined by Eq. 4. 59, which is a function of [A(k))] r 138 (r)1)]rtP 138 [A(V)(r)] a 2nxn submatrix of [A(k)(r)] 153 /u an index 154 ~v ~ the index denoting the set {1,2,..., 2n+1} 153.5i ~ a fixed basis vector in U 47 p a real parameter whose Ip I denotes the value of the components of e Uk2 which are equal in absolute value 146 Op^k a real constant whose absolute denotes the absolute value of the component of the error vector *(k) e U+1 defined by Eq. 2.47 k 56 PM the value of p defined by Eq. 2.50 58 ok -- the real part of sk 10 xxi

LIST OF SYMBOLS (Cont.) Defined or first Symbol Description used on page ~(k) ~ an (n+l)-dimensional vector representing the sign configuration of the error vector *(k) e +l defined by Eq. 2.46 55 (k) (k)''-a vector representing the prescribed sign configuration of the error vector (k) 2n+l 146 - k (C) the 2n dimensional vector representing a prescribed sign configuration of () e U2n 155 e~eU^ 155 0(x) a linear functional of x 50 X an m-dimensional vector representing the sign configuration of x e Em (Eq. 2.34) 50 the initial value of the exponential skt sktI function e, namely e 18 [,] the nx n diagonal matrix whose elements are {1.,, n} defined by Eq. 1.20 18 Wk the imaginary part of sk 10 xxii

LIST OF APPENDICES Page APPENDIX A: STIEFEL'S ALGORITHM 273 APPENDIX B: PROOF OF THE RELATIONS OF LEMMA 4. 1 GIVEN BY EQ. 4. 9 THROUGH EQ. 4.14 299 APPENDIX C: PRONY'S EXTENDED METHOD IN AN m-DIMENSIONAL SUBSPACE, m> n+1 306 xxiii

ABSTRACT This study considers the problem of approximating a prescribed impulse response function, h(t), at a finite number of equally-spaced discrete values of t, by a linear combination of exponential functions, such that the resulting error is minimum in the Chebyshev sense. Specifically, given a set of q values {h(ti)} of h(t), where t. = t + (i-1)At, for i = 1,2,..,q; find the 2n complex constants n skt {a ksk, k = 1,2,...,n, of the function h(t) = a e sothat the Chebyshev error (ti) lo < i < m q h(ti) - h(ti) is minimum when q > 2n. The 2n complex constants { Sk} are limited to a set having the complex sk's and sk's occur in conjugate pairs. If the exponents {Sk} are not distinct, then for each repeated exponent, sk, the apskt skt proximating function possesses terms of the form {e, t e,..,t e }, where j denotes the order of the repeated exponent, xxiv

{ sk}. The approximating impulse response function is such that its Laplace transform can be expressed in rational fraction form. If each exponent, sk, of the approximating function, has a negative or zero real part, then this rational function can be realized as a linear, passive, lumped, bilateral R-L-C network. The discrete time domain approximation problem is formulated in terms of an approximation in the finite dimensional vector space. Such a formulation depicts the relation between the exponents of the exponential functions and the orientation of the approximating subspace. Prony's method for determining the exponents, when q = 2n, is reviewed and is formally extended to the case when q > 2n. The formal extension of Prony's method is used to solve the Chebyshev approximation problem considered in this thesis. A theorem guaranteeing the existence of the best Chebyshev approximation is proved. It is shown by means of examples, that the best Chebyshev approximation'may not be unique. Several properties of the Chebyshev approximation are obtained, including the bounds within which the minimal value of 1ie(t.i)l must lie. It is conjectured that, in general, the best Chebyshev approximation is characterized as that for which at least. (2n+1) valuesof {e (t.)} are equal in absolute value to i e(ti)llI. It is shown, however, that there are some special cases which do not possess this property. xxv

Finally, a computational algorithm for solving the Chebyshev approximation problem is presented, along with some numerical examples. This algorithm determines the optimum a ks and sk s simultaneously. xxvi

CHAPTER I INTRODUCTION:- - X... p- c.s.. i.... The time domain network synthesis problem consistsof finding a practical network which yields the prescribeti impulse response function. The solution to this problem is rarely an exact one because of the following limitations: (1) The resulting network must be physically realizable; and (2) it must employ only a finite number of elements. Thus, the synthesis problem is basically an approximation problem in which the-physically realizable impulse response functions are the approximating functions. This dstudy is concerned-almost entirely with the approximation problem. Specifically, we shall be concerned with."discrete-approximations? where the prescribed impulse response function is approximated at a finite number of equally-: spaced discrete values of the independent variable, time, by physically realizable impulse response functions. In this- chapter, afterdefining the notation used throughout this thesis, we shall present the two steps.which are:essential to the - general approximation problem: (1) the selection of a class of approximating functions, — and (2) the selection of a criterion which measures the degree of the approximation. Then we shall formu- late the approximation problem and present the plan of this thesis. 1

2 1. 1 Notation and Definitions The prescribed impulse response function to be approximated is denoted by h(t), and the approximating function is denoted by h(t; a, s). Both functions are defined for all t in the interval:, where, if not otherwise mentioned, e^denotes the semi- infinite interval [0, c). The ordered pair of n-vectors (a, s) stand for the 2n parametersI of the approximating function,h(t;,s). Tapproximatingfunction h(t; a,s) is selectedfromthe set,:=f{h(t;a,s):.t:e C ae s,:_s c }, where2 s:denotes. the parameter space of the vector a with respect to a fixed- s E:,and 9 denotes the parameter space of. the vector s. A precise defiition of the spaces.s and 9i is given in the, next section.-...- At this -point, it is noted thattheree aetwo diernt: types of approximatons available.- -One is called-the- "continuoustime? approximation: in, which we wish to approximate the; prescribed function for: all t e.; and the other is calledIthe "discrete-time -approximation in which we-wish to approximate the prescribed function only at a finite nu-mber (of discrete values-of L-. t- For our.purpose,:it is -necessary to -dno the parameter s of theapproximating function by the two n-vectors a and s, because the 2n parameter s are -not independentL ispoint will belarified n the next section, when the approximating function will be fully defined. 2 n.'^- " ^..~ ^ ~': -..- - -::.- "".".,..'- -.~:~ -, X..U ^' -~ ~~;; 0: 2The subscript s in s emphasizes the dependence of the parameter vector a on the vector s.

3 In the case of the "continuous-timet? approximation, the approximation error fanction, at any tie t e, will be-denoted by The criterion or measure of the degree of approximation will be given 3 by the L -norm of e (t; a, s), denoted by E (t;a,), where 4 e (t; a,s) is defined by Il~(t;,s)I [ f Ii(.i/p for ipi< oo, and essup E(t;. l Ie (t; a,s) I t0 IE (t; a,t' It suffices to say, that in the case of the "continuous-time" approximation, we are assuming that the error function E(t;a,s) is in the L -space. This approximation will b called the L -approximation. p In the case of the "discrete-time" approximation, we shall be concerned with the values of the functions h(t) and h(t aS) at a finite number of discrete values of t E. We shall denote the discrete See Ref. 21, pp. 212-218., 4More accurately, the following integrai should be a Lebesgue integral. 5See Footnote 3.

4 valu. s of t E:6.,by the finite pdint set-.T {t: i = 1,2,.,q}, and represent theordered set of values (tl), ht2),.... h)} of h(t) and {h(t;a,s), h(t2;a, s),...,h(t;a,s)} of h(t; a,s), by the q-dimensional vectors h and h(a,s), respectively. Hence, in the "discrete-time" approximation, we shall be concerned with rial vectors h and h(a, s) 6 q in the complex q-dimensional vector space Uq. The approximation error vector will be denoted by e (a,s) and defined by ((a,s) h- h(a,s) (1.2) Clearly, thevector:(-, s) must be a real vector in Jq. The measure of the degree of approximation will be given by the -norm of the p vector (a,s), denoted by {Il(a,s)llp, where Ie (a s)lp is givenby ll (a,)l = I I E(, for 1< p < oc i=l and max where E i(a,s) denotes (t.; a,). 6The reason for being concerned with real vectors in Uq, rather than vectors in Eq, the real q-dimensional vector space, will become evident in Section 1. 4, where we shall express the vector h(a, s) by a linear combination of complex vectors. Note that a real vector int Ut is a vector having only real components.

5 Note that in the case of the "discrete-time" approximation, we shall assume that the error vector, e(a, s), is in the real i-space. This approximation will be denoted by the q-approximation. p Definition 1. 1: The function h(t; a *,s) is said to be the best L -approximating function of h(t), defined on., with rep spect to the parameter vector a e s for a fixed s e, if it is selected from the class of functions {h(t; a,s):a e Lds } and satisfies, ll E*(t;s)l]p llh(t)- (t; h(t,)- h(t;a,s)lp for all (a, s) e s x s. The function e *(t; s) is the rees p suiting approximation error function, and a * is the cor-p responding "best" parameter vector in si. For brevity, we shall denote the function h(t;ap*, s) by hp*(t;s). Definition 1.2: The function h(t;a **, s *) is said to be the -P -p best L -approximating function of h(t), defined on oT, p with respect to the ordered parameter vector pair (a,s) cJ, x 9, if it is selected from the class s Y= {h(t;a,s):'a eds, s eb }, and satisfies ll E**(t)llp 11 h(t)- h(t; ap**,*)Ilp < lh(t) - h(t;a,)llp for all (a,s) eds x 9.,dk s The function p **(t) is the resulting approximation -error function, and (e **, s *) is the corresponding "best" ordered vector pair in &l x 9.- For brevity, the function h(t;ap** Sp*) will be denoted by hp**(t)'-

Definition 1. 3: The real vector h(a *,s) is said to be the best jq-approximating vector of a real vector h in Uq with rep spect to the parameter vector a esd for a fixed vector s c P,, if it is selected from the set of vectors {h(a, s)'a e s and satisfies lip*(s)l p 1lh- h(p*,s) i L lp for all (a,s) E S x s. Cd s The vector e *(s) is the resulting approximation error vector, and a * is the corresponding "best" parameter -p vector in. For brevity, the vector h(a *, s) will be des _p co.p - noted by h *(s). -p Definition 1. 4: The real vector h(a **,s *) is said to be the _ -:..P p Q best P"-approximating vector of the real vector h E U with respect to the ordered parameter vector pair (a,s) cg s x 9, if it is selected from the set {h(a, s) a E, s I t } and satisfies L H*Ip = Ih~~a ** s"'~~'s K I.h'-h(a, ~s) ~. I[ep**]lp = _h(_** p**)llp < [!h- h(_as)lp for all (a, s) e x J. The vector E ** is the resulting approximation error vector and (a **,s *) is the corresponding "best" order vector pair in cs x. 9. For brevity, the vector h(ap,**,s *) will be denoted by **.-p —. —-- hN-: -P

7 Remark: From these definitions, it is clear that in general the values of the ordered pair of parameter vectors (ap** s *) change as p is varied, a point which will be further clarified later. Furthermore, it should be mentioned that although we have used the same symbols (a **, s *) to denote the best ordered vector pair corresponding to the best L -approximation of h(t) and the best ordered vector pair corresponding to the best q-approximation of h, the values of p these two ordered vector pairs will usually differ! In closing, it should be mentioned that to simplify our notation, we shall drop the subscript p in a **, sp* hp*(t), c *(t), etc., -p -p p p whenever there is no danger of ambiguity. 1. 2 Selection of the Approximating Function, Exponential Representation The first major problem encountered in obtaining an efficient network with an impulse response approximating the prescribed impulse response is one of selecting the class of approximating functions. In the network synthesis problem, the consideration of physical realizability enters at this point. The requirement that the transfer function of a physically realizable linear, passive, lumped, finite network be a rational function of polynomials in the complex frequency, s, is well-known (Ref. 10). The coefficients of these polynomials

8 must be real; all roots of the denominator'must have negative or zero real parts; and the roots having zero real parts must be simple. Translating these requirements into the time domain shows that for physical realizability, it is sufficient that the approximating impulse response functions have the form n skt -L y'e a k=l where ak and sk occur in conjugate pairs and each sk must have a 7 negative or zero real part. Although this expression implies that 8 the transfer function possesses only first order poles, it should be 9 noted that if the transfer function contains higher order poles, then the approximating impulse response function must possess terms of the form es, t est.., t(j )eSt}, where j denotes the order of the repeated root. It is convenient to define the approximating functions h(t; a,s) by n skt h(t;a,s) ak e (1.3) _ k=1 (l Clearly, the skis represent the pole locations and the ak's represent the respective residues of the transfer function in the complex frequency domain. 8That is, the roots of the denominator, of the rational function of polynomials, are simple. Recall that the poles having zero real parts must be simple.

9 where the collection of the parameters {al, a2,...,a and iS1I s2,...,s } of the approximating function h(t; a, s) are denoted by the vectors a and s, respectively; that is a_.= (1.4) a n and s1 s2 s= ~ (1.5) Ln The number of parameters {ak} or sk} of approximating function is denoted by n. It should be noted that in this thesis, the approximating function is not restricted to the form given by Eq. 1. 3. In otherwords, if the procedure yields repeated values of sk, then for each repeated sk,' stt-eSkt the function e must be replaced by the set of functions es kt t1 ek ^, where j denotes the order of the

10 repeated sk. For example, if s = s2 =... = s,, then Eq. 1.3 is written as s sit n skt h(t;a,s) = (a +a2t+... +.t) e + e (1.3a) - J " ^k=j+l The set in which the parameter vector s lies is denoted by 3 and is defined as follows: Definition 1. 5: The set 3, of the parameter vector s, is a set of all vectors s e Un, the n-dimensional unitary space, with complex components occurring in conjugate pairs, i. e., for each sk = ok +jwk there exists an sk+ = Uk - jw The set in which the parameter vector a lies is denoted by As and is defined as follows: Definition 1. 6: The set X$ of the parameter vector a, is a subspace of the n-dimensional unitary space U which contains all vectors a e U such that when s e P, then the function n st h(t;a,s) = 3a e k=1k is a real function of t. From the above definition, it is seen that the set s depends on the vector s e 3'. Hence, it is convenient to denote the 2n parameters

11 of the approximating function h(t; as) by the ordered vector pair (a, s) and the set in which the ordered pair (a, s)lies by ays x 7. At this point, it is noted that in defining the set P (Definition 1. 5), we have omitted the condition which guarantees that the approximating function h(t; a, s) represents an impulse response function of a stable network, i.e., the condition that Re{sk} < 0, for all k = 1,2,...,n. Hence, the approximation problem in this thesis will not be concerned with the physical realizability of the resulting approximation. 10 It should be mentioned that in the case of the fq-approximation, p the real approximating vector h(a, s) e Uq is taken to be the ordered set of values {h(t; a, s), h(t; a, s),...,h(t a, s)} of h(t;a, s) defined by Eq. 1. 3 (or Eq. 1. 3a). The set in which the ordered pair of vectors (a, s) lies is again the setes/ x 92, where the sets 9P and sd s s are given by Definitions 1. 5 and 1. 6, respectively. One further observation about the selection of the approximating function is that the complexity of the resulting network is directly proportional to n, the number of poles, sk. In summary, then, we shall be concerned with approximating functions selected from the class h = {h(t;, s) a e,s' s e 9 }, where h(t;a, s) is defined by Eq. 1. 3 (or Eq. 1. 3a). 10This point is discussed in Chapter VI.

12 1. 3 Selection of the Criterion of Approximation The second major step toward obtaining an efficient approximation is the definition of a measure of approximation. This measure is generally expressed by the norm of the error function. The two most widely used measures bof approximation in the field of network synthesis are the least-squares, and the Chebyshev (or the uniformnorm) criteria. The least-squares criterion is generally used when the specified data are known to contain random errors. In the case of the "continuous-time" approximation, the least-squares criterion is represented by the L-norm of the error function; that is, 2 r 2 d. 11 (t;-a)s)112- [ I (t;as) dt ]/ L^T = [ Ih(t)-h(t;as)I2dt] (1.6) In the case of the "discrete-time" approximation, the least-squares jq11 criterion is represented by the 1-norm of the error vector, that is, 2 11 (a, ) 2 = ( ea, S), e a, s) ( [h - (Q_,s)], [h - h(as)]) (1. 7) q We use the inner product relation, (e,e), to denote Z e li2, (see Ref. 21, p. 245). i=l

13 The Chebyshev criterion is generally used when a point by point replica of the specified data is desired. In the case of the "continuoustime" approximation, the Chebyshev criterion is represented by the L -norm of the error function; that is, oo e (t; a,s)li = tessup I(t;_a,_s)] = eSup h(t)- h(t;,s). (1.8) In the case of the "discrete-time" approximation, the Chebyshev criterion is represented by the cQ -norm of the error vector; that is, 6I(_, s)1~ = 1<i< q e(a,s)l max l<i<q Ihi (), (1.9 ~ A where i(a,s), hi, and hi(a,s) denote the values of e(ti;a,s), h(ti;a,s), and h(ti; a,s), respectively. In this thesis, we shall be principally concerned with the best Chebyshev approximation of a real vector h e U, and so we shall use the measure of approximation defined by Eq. 1. 19. 1. 4 Formulation of the Problem The problem of approximating a prescribed impulse response function h(t) by a linear combination of exponential functions, - n s kt h(t;a,s) = ak e, at a finite number of discrete values of t, may be formulated as follows: As mentioned before, the ordered

14 set of real values {h(tl), h(t2),...,h(tq)} of h(t), and {h(tl;,s), h(t2;, s).., h(t; a, s)} of h(t;, s) are represented by q-dimensional real vectors in Uq; that is, h(t1) h(t1; a, s) h(t2) h(t2;a,s) h =.; and h(a,s) = t h(t) q;a,) (1.10) q _. sktl skt2 sktq The ordered sets of values {e,e,...,e of the exponens t tial function e, are represented by vectors in U; that is, Sktl s t k 2 e skt e ek e (1.11) where k = 1,2,...,n, and q > 2n. It is evident that if the exponents sk's are complex, then the vectors, ek, are complex vectors in Uq. Furthermore, if the sk's are distinct, then the vectors, ek, k = 1,,..., n, are independent and form a basis of a complex n-dimensional

15 subspace V (s) of Uq. A simple representation of the basis of Vn(s) is given by the column space of a q x n matrix [E(s)] defined by [E(s)] e2 - n] ^ (1. 12) where [E(s)] is of rank n if the set {ek} is an independent set of vectors. Therefore, any real vector h(a, s) in V (s) can be represented by n h(a,s) = ak e (1.13) k=l or, alternately, by h(a,s) = [E(s)] a (1.14) where (a,s) e ca XJ. It should be noted that if the approximating function h(t;a, s) is given by Eq. 1. 3a (i. e., the case in which s1 = s2 =...= sj), then the vector h(a, s) is given by h(a,s) = )(a + 2[T] e + [T] e (1.13a) k=j+l k The notation Vn(s) emphasizes the fact that this subspace is a function of the parameter vector s e oJ, where s represents the ordered set of values {s~}, k = 1,...,n.

16 where [T] is a q x q-diagonal matrix defined by t 0 1 [T] = t2 0 tq Clearly, the vector h(a, s), given by Eq. 1. 13a, can be represented by Eq. 1. 14, where now the matrix [E(s)] is given by e1l tlell.. t e el eln ^11 ^11 ~~~ ^1 11 1,j+1 * n t e 2 t'e t2 e e21 t2e21.~ ti2 "21 e2,j+l e2n [E(s)] = e t e.. et-...e (1. 12a) eql qeql q ql q,j+l. qn where eik represents the i-th component of the vector ek, defined by Eq. 1. 11., Since we have assumed that the vectors { ej+..,} are independent, the matrix [E(s)], defined by Eq. 1. 12a, is of

17 maximal rank n. Hence, in this thesis, the approximating subspace, spanned by the column space of [E(s)], is n-dimensional for all e SP. When the real vector h in Uq is not in Vn(s), then it is related to h(a,s) in V (s) by h =h(a,s) + c(a,s) (1.15) where e (a, s) is some nonzero real vector in Uq. The real vector e (a, s) is the error vector, represented by the ordered set of values e(t;a,S): i =,2,..., q}, of the error function e(t;a,s). Let us suppose that the discrete values of t are equally-spaced at intervals At, so that t. = t + (i- 1) At. Then, each vector e i -k 13 can be written as: 1 zk -k A Zk k- (1. 16) Zk This restriction to equally-spaced sampling points will be discussed in Section 1. 5.

18 skt sA t where Vk = e and = e. Furthermore, the matrix [E(s)], defined by Eq. 1. 12, may be written as [E(s)] = [Z(z)] [I] (1.17) where 1... 1 z z 1 n [Z(Z)].. q- q-1 z1... z (1.18) 1 n z A zn (1.19) and 0 0 4j (1.20)

19 In the case in which [E(s)] is given by Eq. 1. 12a, the matrix [Z(z)] is defined by...-. 1... tjl-l 1... 1..2 [t1 + 2t] j- 1z2 Zl' 1 Z1 Zj+1 Zn [Z(z)] q1.1 -1 -1 q q- Z [t + (q- )At]ij+l. Z n Z (1.18a) Note that if tl = 0, the matrix [Z(z)] of Eq. 1. 18a can be simplified as follows

20 1 0... 0 1... 1 Z1.. Zi Zj+l'z Zn 2 2 j-1 2 2 2 LS Z1 2z 2 Z Z1- Zz Zl I''*' j+ n [Z(z)] = j q- 1 (gq -1j)z1 q- jq- It should be noted that the matrix [Z(z)] defined by Eq. 1. 18a (or Eq. 1. 18b) gives the form of the matrix [Z(z)] when the first j-components of the vector z are identical, i. e., z = z2 =... = z.. Clearly, one can obtain, in a similar manner, a matrix [Z(z)] for any other vector z with components that are not distinct. Rather than obtaining the general form of the matrix [Z(z)], we shall say that when the components of the vector z are not distinct, then the matrix [Z(z)] is defined by Eq. 1. 18a. At this point, it is convenient to replace the parameter vector pair (a,s) of the approximating vector by another parameter vector pair, denoted by (,z); thus, h(a,s), defined by Eq. 1. 14, will be denoted by h(i, z) and defined by

21 h(i,z) [Z(z)] (1.21) where A3 a [ M]~, (1.22) and where the q x n matrix [Z(z)] is defined by Eq. 1. 18 (Eq. 1. 18a). The set in which the parameter vector z lies is denoted by a and defined by: Definition 1.7: The set, of the parameter vector z, is a set of all vectors z e Un, the n-dimensional unitary space, with complex components occurring in conjugate pairs, i. e., for each complex zk there exists a z. = z, k j k' j pk. Remark: It should be noted that the (q x n) matrix [Z(z)], defined by Eq. 1. 18 (Eq. 1. 18a), is of maximal rank n, for all z E. The set in which the parameter vector B lies is denoted byWz and defined by: Definition 1. 8: The set z of the parameter vector, is a subspace of the n-dimensional unitary space U which contains all vectors E un so that if z e, then the vector

22 h(,z) = [Z(Z)]. is a real vector in Uq, where [Z(z)] is the (q x n) matrix defined by Eq. 1. 18 (Eq. 1. 18a). When the parameters of the approximating vector are given by the vector pair (I, z) e z x, then the error vector is denoted by e(J,z). Furthermore, the vector relation of Eq. 1. 15 becomes h - h(i,z) + e(,z) = Z(z)] + E(,z), (1.23) where (fi,),Ez x. 1. 5 Statement of the Problem This thesis will be concerned with the problem of approximating a prescribed impulse response function, h(t), at a finite number of equally-spaced discrete values of t, by a linear combination of exponential functions, so that the resulting error is minimum in the Chebyshev sense. Specifically, we shall be concerned with the following approximation problem: Given a real valued function h(t) defined on the interval [t, tq], select the function h(t; a**,s*) from the class of functions J =- {h(t; as): a egs, s e 9} so that if t e ti = t + (i l): i = ^ 2,, q> 2n, t = [(t-tl)/(q- )]} then

23 max A max l<i<q h(ti) - h(ti s*)l 1< q Ih(ti) - h(ti;,s)l (1.24) for all (a,s) E s xs For the purpose of our investigation, this approximation problem is restated as follows: a 14 Given a real vector h e q, find an ordered pair of vectors4 (**,z*) e z x so that if h** [Zz*)]**, then 11h - h** I0 < ILh - [Z(z)] (1.25) (1.25) for all (,z) egBz x, and where [Z(z)] in the q x n matrix is defined by Eq. 1. 18 (Eq. 1. 18a). Some of the limitations imposed on our time domain approximation problem of network synthesis by this precise mathematical formulation are: (1) The q values {h(ti)} of the prescribed impulse response function, h(t), must be bounded; i. e., max,max'Ih(t)I <. (2) The form of the matrix [Z] dictates that the q values{h(ti)} of h(t) be equally spaced at intervals At, so that t = t + (i- )At. The ordered vector pair (/9**,z*) e gz x b corresponds to the optimum vector pair (a**, s*) ec s x B.

24 (3) There is no guarantee that the approximating function h(t; **, s*) corresponding to the optimum vector pair (**, z*) e z x will yield a network which is physically realizable, even though the prescribed impulse response function h(t) satisfies oc f Ih(t)l dt< o. 0 (4) There is no control on the behavior of the error function, (t) = h(t) - h**(t), for values of t I Te, since the approximation is performed only at the values of t in the finite equally-spaced point set T. e 1. 6 Plan of the Thesis To clarify the role which the theory of approximation is playing in many current investigations in the theory of network synthesis, Chapter II presents a brief summary of the theory of approximation in the language of linear spaces, giving special attention to the singular mappings involved. Furthermore, the previous attempts to apply the theory of approximation to the problem of network synthesis are reviewed. A thorough review of Prony's original work (Ref. 15), and of Ruston's method (Ref. 20) using the Chebyshev norm criterion, are also presented in this setting.

25 Chapter III presents the theory of extending Prony's original work for solving exponential approximation problems in the _q-space. p The theory which leads to the solution of the Chebyshev approximation problem, defined by Eq. 1. 24, is presented in Chapter IV. Special attention is given to the existence theorem and some special properties of the solution are considered. The computational methods leading to the solution of this approximation problem are given in Chapter V. This chapter also contains iterative procedures and illustrative examples which are worked out in detail. In Chapter VI we apply the theory presented in the previous two chapters to the network synthesis problem. Procedure and illustrative examples are also given. A general discussion of the results with recommendations for further study is presented in Chapter VII. In the Appendix we present, in the language of vector spaces, Stiefel's algorithm (see Ref. 22) for finding the best Chebyshev solution to an over-determined system of equations.

CHAPTER II STATE -OF-THE-ART 2.1 Introduction The theory of approximation in normed-linear spaces has been extensively studied by mathematicians for many years as is shown by the many excellent papers and books written on this subject (Refs. 2, 17, 18, 19, 22). In this chapter we shall first attempt to give the reader an intuitive feeling for this subject, so that he may be able to frame the whole problem more clearly. Then, we shall present the previous contributions to the time domain approximation problem of network synthesis. Specifically, we shall review the following contributions: (1) Approximation techniques in the L -spaces; namely, the works of Aigrain andWilliams (Ref. 1), Kautz (Ref. 10), and McDonough (Ref. 13), for p = 2; and the work of Tang (Ref. 23) for p =oo. (2) Approximation techniques in the q-spaces; namely, the P works of Yengst (Ref. 26) for p = 2; and the work of Ruston (Ref. 20) for p = o. 26

27 Let us begin by showing that the approximation problem in the L -space can be looked upon as an'L -projection," or rather as a P p singular transformation, which maps a point (or a function) in an L -space into some point in the approximating subspace of the L - space, so that the distance between these points is minimum in some sense. To illustrate this, we can use the finite dimensional fl —space, since many concepts of the approximation problem in the P L -spaces can be visualized in finite dimensional pq-spaces. p p Recall that any vector in jq may be represented by a linear combiP nation of a complete set of q basis vectors, where q denotes the dimension of the space (Ref. 21, Section 43). Furthermore, any n independent vectors in q, where n < q, span an n-dimensional subspace V of. p We can now state the approximation problem in terms of the " p-projection" problem as follows: Let us suppose that a vector p f and a linear subspace V are given in. Then the vector f* in p V which best approximates f in q with respect to the appropriate p f -norm, is the'" -projection" of f onto V, when p > 1. p p The'l -projection" defined here is a generalization of the P familiar orthogonal projection. The best way of illustrating this is to recall that the familiar orthogonal projection, which represents the best approximation of f in vn onto Vn with respect to the 22-norm, (i. e., the best least-squares approximation of f onto Vn in lq ) can

28 be obtained by placing a q-dimensional sphere at f and expanding it until it touches Vn. The point at which this hypersphere touches Vn is the orthogonal projection of f onto Vn. Furthermore, the radius of this sphere is the f2-norm of the error vector, i. e., 11112 When p > 1, we can, in a similar way, define the 2 -projection of f onto Vn as the point where the smallest q-dimensional convex body Vn. described by IE llp and centered at f touches the subspace V. This 2 -convex body of radius I e llp and center f, denoted by Kp(f, I lip) is defined by Kp(f ll llp) = { l l-f p < ll, in q>}, p > 1 (2.1) A simple illustration of the various shapes of Kp, for p = 1, 2, and oo is given in Fig. 1. In the Chebyshev approximation problem, the 2 -projection of fin gq onto Vn is the point where the smallest qdimensional cube, centered at f and having edges of length 211 11, touches the subspace V. A similar "projection" problem exists when considering the approximation problem in the L -normed linear space, where in the P L -space a function represents a point in the L -space analogous to p P the point in fQ-space described by a vector. Here, however, the p., "L -projection" is more difficult to visualize since both the n-dimensional approximating subspace Vn and the L -convex body are defined in terms of continuous functions in the L -space. Recall that any P

29 2 K K1 g2 -go vn Fig. 1. The best approximation to f onto Vn in E q forp=, 2,oo, where q=2, n=1.

30 1 function f(t) in L (a b)-space may be represented by a linear comp' bination of the complete set of basis functions which is infinite in number since the L (a, b)-space is of infinite dimensionality. p Furthermore, the n-dimensional approximating subspace Vn in the L (a, b)-space is defined by the linear combination of n-independent functions. The L -convex body of radius le(t)lp and center f(t), is p p defined by K(f(t), l(t) = {g(t): lf(t)- g(t)lp < le(t)llp, g(t) in L (a,b)} p' p p p (2.2) For further discussion on this subject, see Rice (Refo 15, pp. 10-14). These concepts are not new in engineering. A well-known example in engineering is the problem of approximating a function f(t) in L2(0, 27)-space by a finite set of sine and cosine functions. In the formal language of linear spaces, this problem may be stated as follows: Given a function f(t) E L2(0, 27) and a finite set of basis functions, {cos kt, sin kt: k = 1,2,..., n} spanning a linear sub2n space V in the L2(0, 27)-space; determine the 2n parameters {alk, k*}' k = 1,2,.., n, so that the function f*(t), defined by n f*(t) ( E (k* cos kt + k* sin kt) (2. 3) k=l The notation Ip(a, b)-space, instead of L (,)-space, is used to emphasize tha the function f(t) is defined for all t in the finite closed interval [a, b], rather than the semi-infinite interval S= [0,oo).

31 best approximates f(t) in L2(0, 27i)-space in the least-squares sense. Since this approximation is taken with respect to the L2-norm of the error function, the solution is given by the orthogonal projection 2n (i. e., "L2-projection") of f(t) onto V. Hence, the 2n parameters {ak*, k*} can be obtained by taking the orthogonal projection of f(t) on the respective basis vectors, namely, ak* = (f(t), cos kt), and (2.4) Ok* = (f(t), sinkt), k = 1,2,..., n (2.5) This is to say, that the set of best parameters { ak, Ok*} given by Eqs. 2. 4 and 2. 5, are equal to the coefficients of the appropriate cosine and sine functions resulting from the Fourier expansion of f(t). This result should be obvious because the infinite set of sine and cosine functions forms a complete orthonormal set of basis functions of the L2(0, 2n)-space. Another evident result is that the sum of the squares of the coefficients of the sine and cosine functions which do not lie in V2n gives the minimal value of the square of the L2-norm of the approximating error function, i. e., I e(t) 1. At this point, it should be mentioned that the best approximation of f(t) onto V n with respect to some other L -norm will not yield the above relationship between the Fourier series coefficient and the best parameters {ak*, *}.

32 In recent years, under the impetus of the increasing need to handle complicated signals, it has become evident that the concepts involved may be best expressed in the language of linear space. For example, considerable attention has been devoted to the problem of representing signals in terms of various bases other than the familiar sine and cosine functions (Refs. 8, 12, 27). Thus, depending on the particular application, the approximating subspace Vn is defined in terms of different types of basis functions. In the field of network synthesis the most efficient approximating subspace is the one spanned by a set of one-sided exponential functions3 -' s, kt. - -' e:t > O, Re{s} < 0, k = 1,2,...,n (2.6) Since the exponents {sk} are usually unknown, the approximating subspace is not fully prescribed, but given in terms of the n-unknowns sk} Hence, the time domain approximation problem of network synthesis involves the determination of the best approximating function on an approximating subspace, Vn, which depends on the set { sk: k = 1,2,.. n}. Recall that in Chapter I we have shown that such an approximation problem involves the determination 2See Section 1.2. 3That these functions are linearly independent when sk's are distinct is obvious.

33 of the parameter vector pair (a*, s*) of the best approximating function h(t; a**, s*) which is selected from the class of functions J = {h(t;_a,s) a e sd, s e'}, where h(t;a,s) is defined by Eq. 1.3 (Eqo 13. a). It, thus, becomes clear that the parameter vector s of the approximating function h(t;a, s) represents the orientation of the approximating subspace vn, a fact which is emphasized by representing Vn by V (s). In the following paragraphs, we shall review the previous contributions to the problem of determining the best parameter vector pair (a** s*) which will be referred to as L -approximation or p _Q-approximation problem depending on the particular criterion used to measure the degree of approximation. In particular, we shall review the approximation techniques that were based on either the least-square, or the uniform norm (i. e., Chebyshev) criterion, since these are the most widely used criteria in the area of network synthesis. It is appropriate to mention here that many of the previous works concentrated on the determination of the optimum parameter vector a e Jis' i. e., a*, for some "good" estimate of the parameter vector s e J, which was considered known for the approximation problem.

34 2. 2 The L -Approximation Problem of Network Synthesis 2. 2. 1 L -Approximations. The first significant application of the approximation theory in the L -spaces to the time domain network synthesis problem can be considered to have been made by Aigrain and Williams (Ref. 1), and by Kautz (Ref. 10). They sought the best least-square approximation of h(t) where t is defined in the interval d = [0,cc). If h(t) and the class of approximating functions are in the L2()-space, then this is the familiar orthogonal projection problem. Here the measure of approximation is given by Ie(t;,s)11 = lf e(t;_,s)I dt (2.7) o oo n skt h(t)- ak e dt (2.8) 0 k=1 k Aigrain and Williams recognized that the optimum ordered vec4.. tor pair (a**,s*) denotes the stationary point of the function lle(t;a,s)ll2 defined in Eq. 2.8. Unfortunately, to obtain this stationary point, one has to solve a system of 2n simultaneous nonlinear equations a task which is, in itself, formidable. Furthermore, this is only a necessary condition, and a unique solution does not neceessarily result. The stationary point x* of a function f(x) is the point where { [af(x)]/[axi]} = 0 for all i = 1,... a.

35 About a decade later, McDonough (Ref. 13) simplified the solution of the system of simultaneous nonlinear equations by linearizing them. He used the orthogonality condition which exists in the Lspace between the approximating function h(t; a, s) and the error function e (t; as), and, thus, determined an expression for IIe (t; a, s)l in terms of the parameter vector s alone. This new function is zero at the stationary point of the function I e (t; a, s). His work on this subject contains an excellent review of other contributions to the time domain approximation problem of network synthesis. A different approach to this problem, developed initially by Kautz (Ref. 10), uses a finite set of orthonormal functions constructed from the set of one-sided exponential functions given in Eq. 2. 6. The choice of the parameter vector s is somewhat arbitrary so that the resulting approximating function is not necessarily the optimum approximating function, h**(t), of h(t) in the L2(T)space. The choice of the parameter vector s is based on Tuttle's 5 interpretation of Prony's work. Tuttle (Ref. 25) realized that Prony's work, which constructs an nt-order difference equation from a prescribed set of 2n equally-spaced values {h(ti):i = 1,2,...,2n} of h(t), can be extended to construct an nth-order differential equation (having constant coefficients) by confining oneself to 5A detailed analysis of Prony's original method is found in Section 2.3.2.

36 the single point t = 0. The important step of proceeding from the differential equation to the approximation problem in the L2(?)space was made by Kautz. However, this method does not usually yield the optimum parameter vector s*. Thus, Kautz's method is basically a two-step approximation procedure. He first determines the pole locations (i. e., the parameter vector s) which are not optimum and then determines the optimum residues (i. e., the parameter vector a) for this pole configuration. The reason for the wide use of Kautz's method stems from the fact that the final approximation error l e *(t)l12 is relatively insensitive to a variation in the parameter vector s. The significant contributions which stem from Kautz's method include: (1) The generalization by Carr (Ref. 5) which extends Kautz's method to the approximation of any impulse response function h(t) c L2()- space and not just those functions h(t) which have derivatives through the n-th order in the L2( )-space. (2) The methods which improve the approximation by changing the pole position (Ref. 4). Kautz's approximation method handles only functions h(t) being everywhere smooth to a high-order in the interval [0, c) of t, i. e., 2the set of functions \: m = 1,2,...,n must be in the Lz(~r)-space. 1

37 2. 2. 2 L -Approximations. The time domain approximation problem of network synthesis using the Chebyshev criterion has not been studied as extensively as that using the least-square criterion. One reason for this is that in Lo (,)-space, the "L -projection" 00 c0 problem is not the familiar orthogonal projection problem defined in L2(,)-space and, thus, is intuitively difficult to visualize. To the author's knowledge, the only contribution using the Chebyshev criterion has been made by Tang (Ref. 23). He shows how to obtain h*(t, s) with respect to the L -norm if the vector s e,9 is prescribed to be a real n-dimensional vector. He, thus, determines the best RC network realization of h(t) in Lc ()-space with respect o0 to only the parameter vector a in is. At this point, we conclude the review of the synthesis techniques in the L -spaces and turn to those in the q-spaces. P P 2. 3 The iq-Approximation Problem of Network Synthesis p The two significant contributions to the problem of network synthesis using approximation techniques at discrete points in the time domain were made by Yengst (Ref. 26) and Ruston (Ref. 20) who measured their approximations by the least-squares criterion and the Chebyshev criterion, respectively. However, in no case, for either criterion, was the optimum pole location (i. e. the vector s) obtained. To clarify this point, we shall analyze the approximation

38 problem in the kq-normed and Aq-normed vector spaces, in rather 2 co complete detail. We shall begin by reviewing the approximation problem when the pole locations of the network are initially prescribed, i. e., the case when s e 9P is initially prescribed when using the formulation given in Section 1. 4. Clearly, when the components of the vector s E P are prescribed to be real, this problem is the usually considered qp-approximation problem in which a real vector in Uq is approxip mated on a prescribed subspace V of Uq. At this point, we note that when the components of s occur in conjugate pairs, and when the approximating vector must be real, then we need to make only a trivial extension to the case in which the components of s are real. Then we shall consider in detail the original work of Prony (Ref. 15) for the case when q = 2n, and interpret the significance of extending this formulation to the case in which q > 2n, in terms of operations in the vector space U. Finally, we shall analyze the works of Yengst and Ruston in the same context. In summary, the specific topics which we shall consider are based on the three forms which the vector [E(s)] a of the equation can take. h = [E(s)] + e(a,s) (2.9) 7The vectors h, [E(s)] a, and e (a, s) of this equation are defined in Section 1. 4.

39 These are: (1) The case in which the vector s e 9S is initially prescribed, but the discrete values { t i = 1, 2,..., q } are not equally spaced. This is the approximation problem frequently considered in the theory of approximation in the q -space. p (2) The case in which the vector s e 9P is not initially prescribed, but the {t: i = 1, 2,..., q} are equally-spaced, q = 2n, and the error vector e(a,s) = 0, so that Eq. 2. 9 can be replaced by h= [Z(z)]p (2. 10) where (, z) e z x. This is the original problem considered by Prony (Ref. 15). (3) The case in which the vector s c is not initially prescribed, but the { t. i = 1, 2,... q} are equally-spaced and q > 2n, so Eq. 2. 9 can be replaced by9 h = [Z(z)] i + E(P,Z) (2. 11) where (f,z) E z x. This is the problem considered by Yengst and Ruston using the least-square and Chebyshev criteria, respectively. Moreover, this case is also the subject of this dissertation. 8The vectors h and [Z(z)] of this equation are defined in Section 1. 4. 9See Footnote 8.

40 It is hoped that this approach helps to unify the material and to set the stage for the main contribution of this thesis which is presented in Chapters III and.IV. 2.3. 1 The Approximation Problem when the Matrix [E] is Initially Prescribed. When the qxn matrix 10 [E] of Eq. 2. 9 is initially prescribed, the approximation problem may be stated as follows: Given a real vector h in Uq and a qxn matrix [E(s)], defined by Eq. 1. 12 (1. 12a) of rank n (n < q), and where s e, determine the vector a * in Js so that ifh* [E] h *, then, lIE*11 - lh lh < II 11h - [E] a i (2. 12) l^*P^-V^^^"P p p p for all a in as and where p = 2, c. This is the form which the approximation problem of Eq. 2.9 takes when the pole location (i. e., the vector s) is initially prescribed and when {ti: i = 1, 2,..., q} are not equally spaced. A problem of this nature, when instead of [E] we have any qx n real matrix [A], of rank n, (n < q), has been discussed in the literature. It can be shown that if the prescribed vector s is any element of,P, then the theorems stated below are simple extensions of those given in the literature for the case Hereafter, we shall denote the matrix [E(s)] by [E] if the parameter -vector s is prescribed and if there is no danger of ambiguity.

41 in which [A] is a real matrix. 2.3. 1.1 The Least-SquaresApproximation Problem. The most familiar approximation problem in the literature is the least-square approximation problem (i. e., the case when p = 2 in Eq. 2. 12). Here the approximation criterion is the ~2-norm oT the error vector E. The existence and uniqueness of the parameter vector a* in es is guarans teed by the Projection Theorem for a finite dimensional Unitary Space (Ref. 6). Furthermore, the vector a* may be determined directly with the aid of the well-known pseudo-inverse matrix (Refs. 7 and 28). These results can be summarized by the following theorem: Theorem 2. 1: For each real f in Uq and each q x n matrix [E(s)], defined by Eq. 1. 12 (Eq. 1. 12a), of rank n, (n < q, s e P), there exists a unique n-dimensional vector a* e s, such that if f* = [E] a*, then le*11 ^ lf -f*ll < llf -[E] all (2.13) - 2 -- - 2 - - 2 for all a f a* in sj. Furthermore, the resulting best leastsquare error vector, * f - f*, is always orthogonal to the best approximating vector, f*, i. e., (f*, *) - 0 (2. 14) o show this, it is sufficient to recall that the approximating vector, - h(a*, s) = [E(s)]cf*, must be selected from a set of real vectors, i. e., if the components of a and s are complex, then they must occur i' conjugate pairs.

42 This theorem is again a trivial extension to the approximation theorem which governs the least-squares approximation in the real vector space, 2. (Ref. 6) The important result of this theorem is that the best least- square approximation of a real vector in U9 by a real vector in the subspace Vn, defined by the column vectors of the matrix [E], is the orthogonal projection of f onto Vn. Hence, the singular mapping involved is given by the projection operator E+: U - Vn, so that the n-dimensional vector o" can be determined from * = [E+] f (2. 15) where [E+] is the pseudo-inverse matrix of [E], defined by [E+] = [E E] [ET] (2. 16) [T where [ET] is the transpose of the matrix [E]. 2. 3. 1. 2 The Chebyshev Approximation Problem. The other interesting approximation is the one that seeks the best Chebyshev approximation of a real vector f in Uq in some subspace Vn of Uq defined by the column space of a prescribed qxn matrix [E(s)], s e,. Here, the approximation criterion is given by the O-norm, i. e., lIE l = max I E. A similar approximation problem, when in0 o0 <i<q stead of [E] we have any qxn real matrix [A], has been discussed in

43 the literature by Stiefel (Ref. 22) and Rivlin (Ref. 18). They have shown that the best Chebyshev approximate solution exists, and they also have developed an algorithm which yields this solution. Furthermore, they have obtained the conditions on the matrix [A] under which the best Chebyshev approximation is unique. Applying their results to 12 the above approximation problem, we obtain the following theorem: Theorem 2. 2: For each real vector f in Uq and each (q x n) matrix [E(s)], defined by Eq. 1. 12 (Eq. 1. 12a), of rank n (n < q, s er), there exists an n-dimensional vector a* ec such that if f* [E] a*, then lle*ll - llf- f*11 = lf- [E]a*ll < Ilf-[E] all (2.17) 0- o - - or- - o - - - - for n-dimensional vectors a e.so The resulting best Chebyshev error vector e* has at least (n+l) components with absolute values equal to II e* l, namely, Ie*I = le*ll, for v= 1,2,..., n+l (2. 18) Furthermore, I* < li HE* for v = n+2,... q Rather than proving this theorem, as is done in numerous places in the literature (Refs. 18, 19, and 22), we shall present the essential Note that again we make a trivial extension to the approximation problem which they considered.

44 steps of the proof in the form which is most suited to our future application. However, first let us offer some intuitive notions relating to the results given by Eq. 2. 18. Consider the real error vector e in Uq given by () = f - [E] (2. 19) Clearly this error vector is a function of the parameter vector a e As', since the vector f and the qxn matrix [E] have been initially prescribed. Let us assume that the function 11 l(a) II is continuous with respect to a. To satisfy Eq. 2o 17, we must determine a vector a = a* e VS so that the 11 (a) I is minimum, that is, le*ll (lle *) l = min ll e(c() ll (2.20) ~ OC ~' 0/ OC That this vector, e*, will have at least (n+l) components with absolute 13 values equal to 11 * II can be illustrated as follows: Select some vec00 tor a = a' e js and determine the vector e(a') from Eq. 2. 19 and the value of lIE(a') II. Let us assume that the jth component of e(a') has the largest absolute value; that is, ej(')l = max Ic (ta') _ Ile(a) 11 (2.21) <i<q -- The following method is called the "method of descent" (Ref. 19) rather than the "method of steepest descent" since we shall not use the maximum gradient of the function It (a) I.

45 Clearly, we can reduce the value of lle(W') 11 (or, equivalently, Ie (a') I) by adjusting any one of the components of a', say a1' until the absolute values of two components of e (a) are equal to 11 (a)11 Let us denote the corresponding vector a by a" and assume that the jt and kt component of e(al") are equal to lE e(a^) H. Hence, we have the relation E-.( " )l = l11le()")ll1o < lle(a')l1 (2.22) We now reduce the value of lle(a") I further, by adjusting two components of a, say ac1" and a2'. This process is continued until we have adjusted all the n-components of a. At this point it is found that the ab14 solute values of at least (n+1) components of e(a) are equal to lle (a) 11 14 If any further adjustment in the components of the vector a increases the value of Ie (a) I1, then, we have achieved the minimum value of 1 e (a) l; i.e., lie* 1 o. The vector a with which we have achieved the minimum value of 11 (a) 11 is the vector a * of interest. c0 The characteristic property of the resulting Chebyshev error vector e * (given by Eq. 2. 18) suggests an alternate method for the solution of the Chebyshev approximation problem. Such a method considers each one of the ( n4) subsets containing only (n+1) equations out of the set of q equations defined by the vector equations in Eq. 2. 19. This method 4That this can be done is evident from the fact that only n components of e (a) are changed independently by varying the n-components of a.

46 is based on the assumption that if, for each subset, the (n+1) components of e have their absolute values equal to some non-negative constant, 15 say Ip, and have their signs chosen to obtain the minimal value of Ip; then, the values of p Ip and the vector a can be determined directly for each subset. 16 Then, out of the set of (n ) possible cs thus determined, one selects the a= a* which corresponds to the greatest 17 Ip. It can be shown1 that this vector a* yields the minimum value of llE[(a)l_, i.e., the vector a* yields the vector c(a*) in Uq with lle (a*) I_ satisfying Eq. 2. 20. This method of solution, sometimes called the "method of ascent", has been studied by Stiefel (Ref. 22) in a geometric setting. Since we shall use some of the results of this approach, let us now use it to sketch the proof of Theorem 2. 2. Let us begin by defining the selection of a subset of (n+l) equations out of the set of q equations, given by the vector relation f = [E] a+ in Uq (2.23) to represent a mapping of the vector space Uq onto an (n+l)-dimensional 15 5The appropriate choice of the signs of the (n+l) components of the vector e in each subset will be given in Theorem 2o 3. 16 That this can be done is evident from the fact that each subset contains (n+l) equations in (n+l) unknowns. These (n+l) unknowns consist of the n-components of a and the one unknown representing the absolute value of all the n+l components of E in this subset. 7See Corollary 2o 1

47 subspace of Uq, where the (n+l)-dimensional subspace of Uq is defined as follows: Definition 2. 1: (Reference subspace with respect to a fixed set of basis vectors.) Let {q.:i= 1,2,..., q} beanorthonormal set of basis vectors of U, so that each vector _ in Uq is defined by q g = Z gi Zi (2.24) i= 1 and let k:j = 1, 2,.., m< ql be a subset of only m of J ^ th these basis vectors, where k denotes the k subset out of the possible (q) distinct subsets. These subsets are arbitrarily ordered, i. e., k = 1, 2,..., (q). Then the m-dimensional is kth subspace, denoted by Uk, is said to be the k reference subspace if it is spanned by the basis vectors in the _k:j = 1,2, kj o.., mn. Furthermore, the projection operator Pk: U - Uk is denoted by a qx m elementary matrix [Ik] and defined by lk] = [5,k 1. ~k2 m1k] (2. 25) -[^ - 1 ^m Hence, the projection g(k) of in Um is related to g in Uq by g(k) = [ 1k] T g (2. 26) An elementary matrix represents a matrix having only one nonzero element in each row and column. This element is equal to one (Ref. 24, p. 96).

48 In a similar manner we can denote, in vector notation, the subsets of (n+1) equations out of the set of q equations defined by the vector equation of Eq. 2. 23. Clearly, using Definition 2. 1, one can form () distinct (n+l)-dimensional reference subspaces, U:v = 1, 2 ~ ("nQ)} from Uq in which the (n+l)-dimensional projections of the vectors f, [E] a, and e in Uq of Eq. 2. 23 are related by f(v) E()] +(), v,2, ) (2.27) where f()= T f UV+ (2.28) [IV] v ] = [I][E] and (2. 29) e(v) [VT +1 (2.30) Let us now summarize the approach which we shall use to sketch the proof of Theorem 2. 2. First, we shall state a theorem which will establish the existence of a unique Chebyshev approximation in an (n+1)dimensional reference subspace Uk1, where k e v:v 1, 2,..., (n1)} o In other words, we shall show that for each real vector f(k) Uk+ and each (n+l)xn matrix [E(k)] of rank n, there exists a unique vector ak* e Hs so that jf(k) - [E(k)] k11 < jf( - [E(k)]ak (2.31) _o [E]- rf

49 for all a = ak* e's Then, using this result, we shall establish the existence of a unique Chebyshev approximation in Uq by showing that the parameter vector a* e's, which defines the best Chebyshev approximating vector f* = [E] a* of the prescribed real vector f E Uq lies in the set a v =1, 2,.., (where a* in Xt which satisfy Eq. 2 31. To establish the existence of a unique Chebyshev approximation in n+1 (or equivalently in U) we need the following assumption: k Assumption 2. 1: Every nxn submatrix of the qxn matrix [E], where q > n, is nonsingular. It should be noted that this assumption will always hold when the prescribed matrix [E] is of rank n and real. 9Furthermore, if [E] satisfies Assumption 2. 1, then every nxn submatrix of the (nx 1) x n matrix [E(k)], defined by [E(k)] 4 [Ik]T[E] (2.32) must also be nonsingular. We shall, henceforth, assume that the matrix [E] satisfies Assumption 2. 1. Let us now present the following lemma by de la Vallee Poussin (Refs. 14 and 18) which we shall use to establish the existence 19See Eq. 1. 12 (Eq. 1. 12a) when the sk's are real.

50 of a unique Chebyshev approximation to f(k) in U1 Lemma 2. 1: For each real linear function 0(x) = (x,y) satisfying the equation 0(x) = c, where x andy are vectors in a real m-dimensional Euclidean space, E, and where c is a real nonzero constant and {yi 0 0:i = 1, 2,.., m}, there exists a unique vector x* in Em, with an m-norm that satisfies IIx*II< tlx 11 (2. 33) 00C 0C for all x x* in Em satisfying 0(x) = c. Furthermore, the vector x* is given by x* aX (2. 34) 20 where sgn y1 sgn Y1X= sgn, and (2 3 5) sgn Y =By definition sgnyi = ifyi ~ and sgn 0 = 00 Yi

51 The proof of this lemma is given in Refs. 14 and 18. However, an intuitive explanation can be given with the aid of Fig. 2, where m = 2. Here the linear function 0(x) satisfying the equation 0(x) = c is a straight line in the 2-dimensional Euclidean space, E2, which is orthogonal to the vector y. Since the points along this line represent the various choices of the vectors x, clearly then, the vector x with a minimal value of 1xII will have components which satisfy: a) Ix11 = 1x21 = lal; and b) sgn x = sgny, i= 1, 2 This vector x has been denoted by x*. Remark: Two points should be noted concerning Lemma 2. 1. First, it should be noted that if c = 0 then the lemma is trivially true and IIx* II = 0 since a = 0. The second point to be noted is that if yj = 0, where j E { i= 1,..., m}, then although Eq. 2. 35 gives the jth component, x.*, of x* to be equal to zero, the equation 0(x) = c can be satisfied by using any value of x.* in the interval [- lx * 11 o lix* 11 o without affecting the final value of Iix* 1oc Hence, we shall say that if yi = 0. for some i, then the uniqueness property of the vector x* fails.

52 2 ~ _ E2-space /1 ~l x"~/ \0(x) =c _01 I1 Fig. 2. Geometric interpretation of Lemma 2. 1.

53 Let us now establish the existence and uniqueness of the best Chebyshev approximation to f(k) in U n+, where k v = 1, 2,. () when the (n+l)xn matrix [E(k)], of rank n, is initially prescribed. Theorem 2. 3: For each real fector f(k) in Un+1 ke {v = 1,2,..., ax ~(k) k,k I (n q)} and an (n+l)xn matrix [E(k)], of rank n (defined by Eq. 2. 32), there exists a unique n-dimensional vector ak* e 6 s so that if f*(k) - [E(k)]] * is a real vector in Un+ then, ll*(k) A i(k) _ f*(k)ll < lf(k) [E(k)]al (2.37) 0 c - o0 0 o-0 for all n-dimensional vectors a ak* e Js. Furthermore, all the o*(k) components of the (n+l)-dimensional error vector e* have absolute values equal to 1ie(k)11 C; i. e., I. (k) 116E*(k)1, i=1,2,.., n+l (2.38) 00 Proof: The proof of this theorem is based on Lemma 2. 1. Let us begin by considering the relation f(k) [E(k)]l+e(k) in U+ (2.39) where the vector f(k) and the (n+l)xn matrix [E(k)], of rank n, are pre(k) scribed; and the vectors a and (k) are unknown. It is evident that the column space of [E(k)], Cn(E(k)), describes an n-dimensional subspace in Uk+1. This implies that the orthogonal complement subspace of Cn(E()) of Uk is simply a one-dimensional subspace. If ^) is a

54 real vector in this one-dimensional subspace, then, ([E(k)] a, x(k)) 0 (2. 40) for all a e 4s; that is, [E(k)]T (k) (2. 41) 21 where (k) l1 (k) (k)= (2. 42) | (k) n (k) n+ 1 (k) st Equation 2. 41 can be solved for (k) with an (n+l)t component arbi22 trarily chosen to be equal to one; i. e., a normalization of the magni(k) (k) tude of (k) is made by taking X = 1o Let us now take the inner product of both sides of Eq. 20 39 with re(k) spect to x(k) This yields 21 (k) It should be noted that the vector ) determined from Eq. 2.41 will always be a real vector because we have assumed that if the column vectors of the matrix [E]k)] are complex, they must occur in conjugate pairs. 22Note that, by Assumption 2.1, xi(k). 0, for all i = 1,2,...,n+1.

55 ~(f^ ) - ()(2. 43) (f(k) (k)) = (e(k) (k)) (2.43) since ([E(k)] a, x(k) = 0 from Eq. 2. 40. Furthermore, the vectors f(k) and k(k) are both real and known, then Eq. 2. 43 has the form (E(k), (k)' = ck (2. 44) 23 where (f(k) (k)) (2.45) k The inner product ( (k), (k)) is a linear functional of E(k) Let us represent it by A(E(k)) A (k) 0(E ) (E(k), (k) (k) The totality of all points (k) satisfying the equation E (k)) = ck is a n+i hyperplane in the space Ek. The problem now is to determine the point e*(k) in the hyperplane, (e(k)) = ck, so that lIE*(k) l is minimum. y = (k) From Lemma 2. 1, the minimal value of (k)I is attained -. _ i -t-noc when (k) = *(k), where E*(k) is defined by e*() k k(k) (2.46) tvtI23sl ftk) andg sb (Pk - If ck = 0 (i. e., the vectors f_(k) and o(k) arerthogonal), then the vector f(k) must lie in the approximating subspace, Cn(E(k)).

56 and where Ck C (2. 47) k - ^(k) j and sgn x 1(k) _(k) sgn 2(k (2.48) (k) sgn n(k _*(k) Knowing*(k) and substituting it into Eq. 2. 39, will give the vector a*. Thus, the theorem is proved. This theorem is illustrated geometrically in Fig. 3, where U+1 is taken to be a real 2-dimensional space, E2. The prescribed vector f(k) and the prescribed column space of the (2x 1) matrix [E(k)] are depicted by the vector f and the line C (E), respectively. The vector X, which lies along the perpendicular to the line C1(E), represents the (k) vector(k) which satisfies Eq. 2. 41. In the figure, we show X to be directed in the first quadrant. 24 The straight line, denoted by 0(e) = c, represents the relation of Eq. 2. 43; namely, the line along which the orthogonal projections of f and E onto X are equal. The dashed straight 24If is directed in the other direction, then ck < 0 in Eqs. 2.45 an 2. 47. However, this does not change the procedure.

57 2.X 2 _ i / a )E -space \ \ - ier C1(E) Fig. 3. Geometric interpretation of Theorem 2.3.

58 lines (at 450) represent the direction of the vectors E having components equal in absolute value (see Lemma 2. 1). Clearly, the vector e with H1 II minimum and satisfying Eq. 2. 43, must be the vector e* shown in the figure. The vector f*, represents the best Chebyshev approximation and is given by the difference f - e * In summary, Theorem 2. 3 establishes the existence of a unique vector a * e s which defines the best Chebyshev real approximating vector f*(v) [E(v)] * to the real vector f(v) in U The vector v v f() and the (n+l)xn matrix [E()] v= 1, 2,..., (n) are obtained from the vector equation f = [E]a+e in U (2. 49) The next step in the proof of Theorem 2.2 is to show that the above results imply the existence of a unique vector a* e as which defines s the best Chebyshev approximating real vector, f* = [E] a * to the prescribed real vector f e Uq and the prescribed qxn matrix [E]. Let IPMl be the largest value out of the set lpvI:v= 1,2, ooo, (n+q ) where Pv is defined by Eq. 2. 47; that is, A IpM max {Ipvl:v= 1,2,.., (n)} (2.50) v Consider the relation llf - [E]IIc_ < IPMI

59 or alternately, n -IPM I fi - e a. < II i 1,2,... q (2. 51) -. - where f: i = 1,2,... qo represents the q-components off e U, and ei: i = 1,2,..., q; j = 1,.., n represents the elements of the qxn matrix [E]. The point sets in n-space, defined by Eq. 2.51, are convex sets. Recall, from Theorem 2.3, that any (n+l) of these convex sets have a point in common, namely a * e cs' where v = 1, 2,.., ( n1) Hence, by Helley's Theorem (Ref. 16), there exists a point which lies in all the convex sets of Eq. 2. 61. This point, a*, is equal to aM* which corresponds to the value of IPMl. Furthermore this a* is a unique vector in's, since, by Assumption 2. 1, any other a * E S', v = 1,2,.., ( q1) violates Eq. 2. 51 for at least one i E i = 1,2,..., q} This completes the sketch of Theorem 2.2. We can now state the method which yields the best Chebyshev approximating real vector f* = [E] a* to a real vector f in Uq by using "the method of ascent" as follows: Corollary 2. 1: Let f* 4 [E] a* be the best Chebyshev approximation to f e Uq, where [E] is a given qxn matrix of rank n, defined by Eq. 1. 12, and let f*(v) [E(V)] a * be the best Chebyshev approximation to f(V) e Un+l, where the matrix [E(V)] is a (n+l)xn matrix,

60 defined by Eq. 2o 32, f(v) is defined by Eq. 2.28 and v = 1, 2,.., (n1). Then, there exists an (n+l)-dimensional reference subn+ *s space denoted by U so that the vector a * of the best Chebyshev C "C (c) * to f(c) U n+ i approximation f*(c) [E)] a * to f() U is equal to the vector a* of the best Chebyshev approximriation f* - [E] a* to f e q; and e*(C)ll ^lf() [E(e)] a* = le*ll 1 II- [E] *a*l( ~C*C - - co - - oc (2. 52) where f(c) [E()] a*, and e *() are (n+1)-dimensional vectors in n+ 1 n+ 1 U. Furthermore, this reference space, U, is one which c c maximizes the deviation [ e*(v)1 _ of the best Chebyshev approxi(v) n+1 mation to f in U among all the (n+l)-dimensional subspaces V {Un:v= 1, 2, (nq); that is, v n+1 v *(C) > >'le*(V),v= 1,2,.o, n(2.53) lie*(C) Io > I *(v) II vI_, 2 1 I2 11^*^110 n \+( ) (2 53) The proof is along the same lines as the sketch of the proof of Theorem 2.2 (see Ref. 19, p. 66). This corollary states that the problem of minimizing the value of llI (a)ll If- [E] a11 with respect to ae s, that is, Ie* 1II = min 11e(a) I (2.54) — oC,,~,s

61 where f, [E] a, and e are vectors in Uq, may be replaced by the problem of determining the largest value of Ile (va*)llC from the set l lI(V)(a *)I1 llf(v) [E(v)] - *:v 1,2, ~ nl) that is, ie*1l =max lI(v *)l v = 1,2,.., (ql) (2.55) where [E()] a * is the best Chebyshev approximation to fv) in U -V -V We are now in a position to illustrate a method of computing the best Chebyshev approximation to a real vector f in Uq, based on Corollary 2. lo One merely computes the set {av*', which defines the best (V) (V) approximation of f on C (E) in all the (n+l)-dimensional reference subspaces {U }, v= 1,2,.., ( (q), and then chooses the one a * which yields the largest deviation ( *) hich yields the largest deviated in 4, where is assumed to be a real 3This is illustrated in Fig. 4, where Uq is assumed to be a real 3dimensional space, E3. The vector f and the line C (E) represents the prescribed vector and approximating subspace, defined by a (3x 1) matrix [E]. The three distinct 2-dimensional reference subspaces U2, where v = 1,2,3, are the three coordinate planes 1-2, 2-3 and 3-1, respectively. The projections of f and C1(E) on the coordinate planes are depicted by f (V) and C 1(E(v)), respectively, where v = 1,2,3. The

62 cube shown represents the smallest cube which can be fitted between f and C (E), i.e., its edges are of length lIe* II. By comparing the projection of this cube on the coordinate plane and the smallest square of length IIe( (v *) II which can be fitted between f (v) and C (E(V)) for all v = 1, 2, 3, it is seen that the largest square is obtained when v = 1. There are methods of selecting the appropriate (n+l)-dimensional subspace U nout of the set of Un+l which are more systematic 25 than the random search described above. One of these, because of Stiefel (Ref. 22), uses a point-by-point exchange procedure which sys26 tematically converges to the solution. Another algorithm, because of Remez (Ref. 17), which converges faster to the Chebyshev approximation, is based on exchanging all the points at each step. At this point, we conclude our review of the approximation problem when the pole location is initially prescribed; i. e., the vector s in,P of the matrix [E(s)] is prescribed. However, before considering the contributions to the approximation problem when s is not prescribed initially, we shall review Prony's original work. We do this for two reasons. First, to introduce the formulation and illustrate how the constraint which requires that {t.: i = 1,2,.., q} be equally-spaced, simplifies the mathematics. Second, to demonstrate that when q = 2n, 25 2 See Appendix A. 26The reader is referred to the treatment of this subject by Rice (Refo 19, pp. 176-180),

63 2 C (EC ) f(1) 1 \\^f=P' 1 3 1/ 2 3,^^3' Fig. 4. Approximation of f in onto C 1(E) in the Chebyshev sense. \(2) i[l/P 3 Fig. 4. Approximation of f in E3 onto C l(E) in the Chebyshev sense.

64 we can obtain an exact representation of any real f in Uq, i. e, when q = 2n, then in general e = O. 2. 3.2 The Original Method of Prony Prony (Refo 15) solved the following exponential interpolation problem: Given a set of measured data points of a process; determine the interpolation the values of the intermediate points when the behavior of the process is governed by a linear combination of exponential functions. In other words, he sought the values of the parameters {ak,, sk k = 1,2,. o, n, of n s t f(t) = e (2. 56) k=1k when the set of 2n points {f(ti)} were specified. To carry out this interpolation, he suggested that the 2n points {ti be equally spacedo Hence, he determined the 2n unknowns {ak, Sk}' k = 1,2,..., n from the system of 2n simultaneous equations given by n skti f(ti) = k e i=1,2,..., 2n (2.57) i k=l k where ti = t1 + (i-1) At, and At is the interval between the equally spaced {ti}. At this point he observed that since there are 2n unknowns in Eq. 2. 56 which must satisfy 2n independent conditions, an exact solution is possible. Now before turning to a detailed consideration of Prony's method

65 of solution, we observe that if the 2n unknowns {ak, k} must satisfy q conditions, where q > 2n, then Eq. 2. 57 represents a system of overdetermined equations. Since, in general, an exact solution to such a problem cannot be obtained, then one must seek an approximate solution, by using approximation methods. The extension of Prony's original method to finding the approximate solution of such a system has been attempted in the literature and is sometimes misnamed, "the Prony method. To avoid this misconception, we shall refer to Prony's initial interpolation method as "Prony's Original Method, " and the approximation method based on this method as "Prony's Extended Method. 27 In this section, we shall review "Prony's Original Method" in terms of operations in the vector space Uq, when q = 2n. It is hoped that this approach will aid in clarifying the limitations of the previous works (discussed in Section 2. 3. 3) in which this method is used to solve approximation problems of network synthesis. Clearly, Eq. 2. 57 may be stated in vector notation using the formulation of Section 1. 4 as follows: Given a real vector fain U2, determine the ordered pair of parameter vectors (j, z) in'z x so that 27 The'Prony's Extended Method" should not be confused with the previous methods (to be discussed in Section 2. 3. 3), which use the approach of'Trony's Original Method" to solve approximation problems, The formal definition of "Prony's Extended Method" will be given in Chapter III.

66 = [z(z)] (2. 58) where 1.. 1 Z1 zn I i^l t [Z(z)] = z z z =, and f = 2 -... ~ 2 The essential contribution ofProny is in the method he employed in finding the vector pair 3, z). He observed that if the (2nxn) matrix [Z] is of rank n, he could replace the 2n unknowns z, k = 1, 2,.., nby a new set of n unknowns rJ i=O,.., n-i, which linearly relate any (n+i) successive rows of the matrix [Z]. Furthermore, this th relation may be expressed by the n -order polynomial equation n-. n Pn(z) zn + i r. Z = (z-z = 0 (2.60) ^ -O k=l 28 Note that the matrix [Z(z)] defined here, represents the matrix defined by Eq. 1. 18, where q= 2n, that is, it illustrates the form of [Z(z)] when the components of z e < are distinct. If the components of z are not distinct then [Z(z)]takes on the format of Eqo 1. 18a," where q = 2n.

67 the roots of which are the n-unknowns {Zk}. Hence, this method consists of first determining the n unknowns {ri}, i = 0, 1,.., n-l, and then determining the 2n parameters, 3k' Zk}, k = 1,..., n. This is illustrated as follows: If the 2nxn matrix [Z], defined by Eq. 2. 59, is of rank n, then the format of [Z] reveals that any n consecutive rows of [Z] are independent. Hence, any (n+l)st row can be represented by the linear combination of the previous n rows, that is n+v-1 v+i-2 k= 1,2,..., n (2.61) z C. z, (2.61) k i=1 i=l v = 1, 2,..., n Observe that the set {Zk:k = 1, 2,..., n} represents the n-roots of th the n order polynomial equation given by n n zn - Cz = II (z-z) = 0 (2.62) i=l k=1 Hence, if we let c. = -r.i, then Eq. 2o 62 yields the polynomial equation given by Eq. 2. 60, and Eq. 20 61 may be written as zv{Pn(Z)} =, v= 1,2,..., n (2.63) th where P (z) is the n -order polynomial defined by Eq. 2. 60. At this point it should be noted that the polynomial Pn(z) is invariant of the index v in Eq. 2. 63. Furthermore, by specifying the

68 n-coefficients {r.: i = 0,,.., n-l} of Pn(z) one also specifies the elements of the matrix [Z(z)] defined by Eq. 2. 59, since the set of values {z k = 1, 2,..., n} can be determined directly from Eq. 2. 60. We now have the problem of determining the values of the n- coefficients { r: i= 0, 1, 2,..., n-l} of Pn(z) from the 2n prescribed data points {f: i = 1, 2,..., 2n}. Recall that since the set of n-unknowns {ri} linearly relate any (n+l) rows of the matrix [Z(z)], then by the relation of Eq. 2. 58 they must also linearly relate any (n+l) successive components of the vector f, that is, n-1 fv+n + ri f+v = v= 12,..., n (264) i=0 Clearly, Eqo 2. 64 represents a system of n simultaneous linear equations in n-unknowns. If these equations are independent, then they yield a unique set of { r.}, which are real since the { f.} are real. Knowing {ri}, the {Zk} follow, being the roots of Eq. 2.60. (Note that since the {ri} are real, then the roots, {Zk}, of Eq. 2, 60, are real or occur in conjugate pairs. ) If all the roots of Eq. 2. 60 are distinct, then the { k} are determined from Eq. 2o 58, where only the first n-relations need be considered. On the other hand, if some of the roots of Eq. 2o 60 are repeated, then Prony shows that the {k} can be " determined by using a modified form of Eq. 20 58. This modified form s t of Eq. 2. 58 is obtainedfrom Eq. 2, 56, by replacing the function e,

69 s t for each repeated root of Eq. 2. 60, by the function t e where j denotes the order of the repeated root. For example, if zl = z. -th = z. represents the j order root of Eq. 2. 60, then Eq. 2. 57 is written 29 as sti n s ti f = (1 + a2 ti +' + a.t.j-) e + ak e ii ~ ~k=j+l i= 1, 2,.., 2n Since t. = t1 + (i- 1)At, then, if t1 = 0, this expression yields the following modified form of Eq. 2. 58, f. = [1 +(i- 1) At2 +.. + (i-l)j'Atj -l] z, + P kZk k=j +1 i= 1,2,..., 2n This is illustrated in the following example: Example 2. 1: Let us fit n = 2 exponential functions to the function f(t) = 3- t at the following 2n discrete points t = 0, 1, 2, 3, i. e., At = 1. In other words, given the vector 3 2 Sf- ql 29See Eq. 1o 13ao

70 find the vector pair (_3, z) e z x 3 when n = 2. Equation 2. 64 yields the following sets of equations 3r0+ 2r1+ 1 = 0 3r0 +r1 = 2 r0 + 1 = 0 Solving this system of equations, yields r0 = 1, r1 = -2. Hence Eq. 2.60 becomes 2 - 2z + 1 = 0 the roots of which are z1=, z2= 1. Since we have a double root, i. e., z1 = Z2 = and since t1 =0 and At = 1, then from Eq. lo 18b the matrix [Z(z)] becomes 1 0 [Z(z)] = 1 2 1 3 Solving for the vector 3 which satisfies Eqo 2.58, that is, 3 1 0 2 ^ 1 1 1 2 0 1 3

71 one obtains that 3 = [3, -1]. Therefore, the vector pair (i, z) is given by (_, z) = (- i [U 30 Mapping the vector pair (3, z) into (a, s) yields (a'-) ([1 [O)) Hence, the interpolating function, given by s1t f(t;,s) = [a +a2t] e becomes f(t) = (3- t) Clearly, the interpolating function f(t) does indeed pass through all four sample points. In fact, since the function f(t) is identical to f(t), the 31 interpolating function f(t) will pass through any q-sampling points, where q > 2n= 4. At this point, we recast Prony's method in terms of vector operations in U. First, let us redefine the polynomial of Eq. 2. 60 as 30This is discussed in detail in Section 6. 2 of Chapter VIo Note that this problem illustrates the case when the approximation problem yields a zero erroro

72 n P(z) A ri z (2.65) Clearly, when rn 0, then the zeros of this polynomial are identical to n th the zeros of the n order polynomial P (z).~ Hence, the choice of rn = 1 n n in the definition of P (z) is arbitrary. If we denote the ordered set of coefficients {r0, r1,, r by the vector r En+ ie., r r r = (2. 66) r n then it suffices to say that Prony's method seeks only the direction of r E E, and not its magnitude. In other words, Prony's method seeks the vector r E En+ which is restricted to the set defined by Irll = 1. _- p The implications of this formulation will become evident in the next sections, where the extensions to the "Prony's Original Method" are discussed. For the present discussion we shall follow "Prony's Original Method" and assume that r = 1. Let us now represent the set of n polynomial equations of Eq. 2.63 in vector form, namely, [Z(z)]T(v) = 0, v= 1,2,..., n (2.67) where the 2n-dimensional vector r(V) is given by

73 0 0 th r0 - v components 0 r(V) rn 0 It is evident from Eq. 2. 67 that the set of vectors {r(V)}, v =2,.., n 32 are orthogonal to the column space3 of [(z)], Cn(Z); that is ([Z(z)],r v) = 0, v= 1,2,..., n (2. 68) for all / e. In fact, the set {r(), v = 1,2,..., n, spans the orthogonal complement subspace of C (Z) in Un. In matrix form, this orthogonal complement subspace may be represented by the column space of a 2nx n matrix, [R], which is defined by 32Recall from Section 1. 4, that the matrix [Z(z)], defined by Eq. 1. 18 (or Eq. 1. 18a), is of maximal rank n, for all z e.

74 r0 0... 0 0 rI r0 r2 r 0 * 0:2 r [R] A [r(1) (2) r0 (2.69) n 2 1 0' r2 O' r. n 1 0 0.oo 0 r We have thus defined an n-dimensional subspace, C (R, which is a function of the (n+l)-dimensional parameter vector r in E, defined by Eq. 2, 66, Furthermore, this subspace is orthogonal complement 33 2n subspace of C (Z) in U n In order to satisfy the equation f = [Z(z)], the vector f must lie in Cn(Z), or alternatively f must be orthogonal to C (R); that is [R]Tf = 0 (2 70) Clearly, when r = 1, this expression represents the system of simultaneous linear equations of Eq. 2o 64. Observe that Eq. 2.70 may also be written as 33. This will be discussed further in Chapter III.

75 [F]r = 0 (2.71) where fl f2 ~ fn+l f2 f3 fn+2 [F] = 2 n2 = nxn+1) matrix f f f n n+l I 2n This equation can be solved for r when one of the components of r is arbitrarily chosen to be equal to one. For example, if we select rn = 1, and if the first n columns of the matrix [F] yield an nx n nonsingular matrix, then the other n components of r can be obtained from r f* oI n n+l (2.72) r f f f ^n-1 n 2n-1 2n In conclusion, then, we have shown that Prony's original contribution consists of determining a new parameter vector r which is related to z by Eq. 2. 65 and defines an n-dimensional orthogonal complement subspace to C (Z) in U. Using this setting, we are now in a position to summarize the contributions by Yengst and Ruston to approximation methods for network synthesis.

76 2. 3. 3 The Approximation Problem when the Matrix [E is Not Initially Prescribed. We shall now consider the previous contributions to the problem of network synthesis using discrete approximation techniques. This is the case described in Eq. 2.9 when the vector s e 9P (i. e., the q x n matrix [E(s)]) is not initially prescribed and when q > 2n. The two significant contributions towards the solution of this problem were made by Yengst (Ref. 26) and Ruston (Ref. 20) who employed the least-squares and the Chebyshev criterion, respectively, to judge the degree of approximation. However, in no case, was the optimum pole location (i. e., the vector s* e 92) determined, Both Yengst and Ruston used Prony's approach to solve their respective approximation problems. Therefore, their methods require that the discrete values of the prescribed impulse response be taken at equally-spaced points of time. Although they did not formulate their problem in terms of the theory of approximations in Uq, it can be shown that they sought the best vector pair (3**, z*) e x B which minimizes the q-norm of the error vector (io e., lle(P, z)II with p = 2 for p p Yengst's problem and p = oc for Ruston's problem) of the equation h =[Z(z)] + e(3, z) in Uq (2. 73) where q> 2n. Their methods of solution involve a two-stage approxi-mation process, by which they first determine an estimate of the pole location (or equivalently, the parameter vector z) by extending Prony's

77 original method; they then determine the optimum residues (or equivalently, the parameter vector P*) for this pole location. Now before turning to detailed consideration of their methods of solution, let us show that their approximation procedure yields an error norm that is not necessarily the minimum possible. Recall, that when q = 2n, "Prony's Original Method" yields a system of n simultaneous equations, given by Eq. 2o 70; i. e., [R(r)]h 0 in En (2. 74) This system can be directly solved for the vector r, They argued that when q > 2n, one obtains a similar system of equations; however, now Eq. 2o 74 characterizes an overdetermined system of equations since [R]T h is a (q-n)-dimensional vector, where (q-n) > n.o Being thus denied an exact solution, they sought the best approximate solution to this overdetermined system of equations, i. e., the vector r which satisfies the vector relation [R(r)] h = (r) in E-n (2. 75) when I611 is minimum. By applying the operator [R]T to Eq. 2o 73, 34 To emphasize the dependence tof the vector i on r, the vector _(_, z) is denoted by e(r)o

78 (r) = [R(r)]TE(r) in E-n (2.76) Clearly, minimizing the function 116(r) 11 with respect to r does not necessarily minimize the function IIE(r)llp. Therefore, in general, the vector r, which represents the solution vector r when 116(r) II is minimum, will not be equal to the vector r*, which represents the solution vector r when IE (r) II is minimum. p Furthermore, it should be mentioned that Ruston's and Yengst's approximation procedure considers only the vectors r e E with the (n+l)st component equal to one, io e,, rn= 1, rather than the vectors n+1 3 5 r 6 E which lie in the set defined by llrll = 1. Hence, they minimize the function 11[(r')1p with respect to r e En, where the vector r' is defined by r r' = 1 (2. 77) r n-I The consequence of this assumption will be discussed further in the next sections, where their individual works are considered. 35 See Eq. 2o 66 of Section 2. 3.2.

79 2.3.3. 1 The Leaste-Square Approximation Method of Yengst. Yengst (Ref. 26) considered the network synthesis problem using discrete least-square approximation techniques in the time domain. This problem is stated by Eq. 2. 73o By using "Prony's Original Method" and the general least-square approximation theory, he was able to obtain an approximate solution to this problem. In the setting introduced above, Yengst determines the n-dimensional vector r', defined by Eq. 2. 77 which satisfies Eq. 2. 75 in the best leastsquare sense; that is, 16(r') II ll[R(r)]T hll < II[R(r')]Thll (2.78) ~ ~ 2 ~ ~ 2 ~ ~ 2 for all r' in E. To elucidate the dependence of the vector 6 on the vector r', let us rewrite Eq. 2.75 as 6(r') [Hn] r + h (2.79) where the (q-n) xn matrix [H] and the (q-n)-dimensional vector h are defined by h hh h... n n+l H = h2 hn+l,and h = n+2 (2. 80) n |n 0. h q h. h - q-n q-l, q

80 Since the I2-norm of 6 is given by 115(r')I}2 = ([Hn] r+h [H] r'+h ) n -n -n - -n then, it follows from -- 11(r) 0 that rr' a T T [HT H] r' = -H Th- (2.81) Jn n -n -n This expression is the vector equivalent of Eq. 32 of Yengst's paper, (Ref. 26). The vector r' represents the stationary point of the function l_(r')2. If the nxn matrix [Hn Hn] is nonsingular, r' can be determined from r [HT Hn] Hn n (2.82) Having determined r', Yengst proceeds to determine the vector z in terms of the roots of the polynomial equation n-1 n z + rz = I (z- k) = 0 i=0 k= From the above, it is evident that the resulting pole locations (or equivalently, the parameter vector z) depend on the stationary point of the function H_ 6(r')II L( = ) [Rr (r )] (') 11 rather than on the stationary 2 -2 point, r* of the function Ile(r)l 1. Hence, Yengst's approximation technique does not necessarily yield an optimum least-square approximation to the prescribed real vector h in Uqo

81 Furthermore, since Yengst initially assumes that the (n+l)st component of r e E is equal to one, then the vector given by [r11, r2'.., rn' 1] may not represent the vector r e En+l which r 2 n' minimizes the function 116(r)1I along the set defined by lr Ip = 1, when 6(r) is defined by Eq. 20 75. This may be best illustrated by defining the n+ vector 8(r) of Eq. 2.75 in terms of r E E, namely 6(r) = [H] r (2. 83) where h h - *hn+ 1 [H] h ~ h"2 h o 0 h q-n q If one initially assumes that some other component of r is equal to one, then Eq. 2. 83 clearly yields a relation which differs from Eq. 2. 72 only by the value of the elements of the matrix [Hn] and the vector h. For example, if the component r0 = 1, then Eq. 2. 83 may be written as (r") = [H] r" + h (2. 84) where

82 r hq- n and h2 hn+ 1 h h [Ho] = 3 h n+2 = (q-n)xn matrix ~ 0 ~ 0 h h q-n+l q Note that while the vector h0 represents the vector to be approximated in Eq. 2. 84, it represents a vector which spans the approximating subspace, defined by the column space of [Hn], in Eq. 2o 79. To illustrate that the relations given by Eq. 2.79 and 20 84 may yield two different best least-squares solutions (after making a normalization with respect to ilrll ), let us suppose that in Eq. 2. 84 the vector h lies in the column space of the matrix [H0]. Clearly then, Eq. 2. 84 may be solved exactly, so that the minimal value of 11 (r)11n2 = 0o On the other hand, Eq. 2. 79 cannot be solved exactly (if the vector h does not lie in — n the column space of [Hn]), so that the minimal value of 116(r')12 / 00 n+ 1 Hence we note that, for this case, there exists an r e E along the set Iri p = 1 which yields a 116(_)112 = 0, where 8(r) is defined by Eq. 4. 84, a result which is not obtained by using Yengst's formulation of Eq. 2. 79.

83 2. 3. 3.2 The Chebyshev Approximation Method of Ruston. Ruston (Ref. 20) considered the Chebyshev approximation problem of network synthesis defined by Eq. 2.73. His method of solution is similar to that used by Yengst. By using "Prony's Original Method", and Stiefelts work (presented in Appendix A) he was able to obtain an approximate solution to this problem. His method involves a two-stage approximation where he first determines the parameter vector z and then determines the optimum parameter vector /* for this z. His procedure leads to an approximation which gives a Chebyshev type error, but as will be illustrated in the next paragraph, it is not the minimum possible Chebyshev error. To determine the parameter vector z, Ruston begins by finding the n-dimensional vector r which satisfies Eq. 20 75, so that I6(r"') ll - lI [R(r')T hll < l [R(r)T h l (2. 85) for all r' in En. Actually, to determine r' he uses the alternate representation of the vector 6(r') given by Eqo 2. 79; that is, (r') = [Hn] r' +h (2 86) Since both the (q-n)xn matrix [Hn] and the (q-n)-dimensional vector h are known (see Eq. 2.80), the vector 6 is strictly a function of the n-dimensional vector r', defined by Eq. 2.87. Hence, if (q-n) > n and [H ] is Efrn of rank n, then the problem of finding the vector r' E E which minimizes

84 116(r')l o, where the vector 6(') is defined by Eq. 20 86, is simply the Chebyshev approximation problem considered in Section 2. 3. 1. 2. The existence of the best Chebyshev approximate solution vector r' e En is given by Theorem 2. 2, where now that the vector h and the -n n columns of the matrix [Hn] describe vectors in a (q-n)-dimensional Euclidean vector space, E q n At this point, Ruston applies Stiefel's algorithm to obtain the' e En which satisfies Eqo 2. 85. Once the vector r' is attained, then Ruston determines the parameter vector z from the roots of the polynomial equation n-i P (z) = + r. z 0 (2.87) ^n j0i=O Let us denote the ordered set of roots of Eq. 2. 87 by the vector z. Consequently, from the vector z, the qxn matrix [Z(2)] can be fully determined. Denoting this matrix by [Z], he arrives at the second approximation step which is defined by the equation h = [Z] + (2.88) where both the real vector h in U* and the qxn matrix [Z] are known. This step seeks the best Chebyshev approximate solution vector 3* e z z so that lE 11 is minimum. Clearly, the existence of a solution to this problem is given by Theorem 2. 2, so that Ruston again applies Stiefel's algorithm to determine the vector If* E f. It should be noted that the """ z

85 resulting error vector, given by 6*(z) = h- [Z] 3* (2.89) will satisfy the conditions given by Eq. 2. 18 of Theorem 2. 2; i. e., that the absolute values of at least (n+l) components3 of e*(z) are equal to le_*(z)llO_ and the absolute value of the others is less than Ile*(z)iil. 37 To illustrate Ruston's method, consider the following example:37 Example 2.2: Given: q = 9, n = 2, and 1. 0000 0. 4450 0.2500 0.1600 h 0.1110 0. 0817 0. 0625 0. 0494 0. 0400 Find the vector pair38 (3*, z) e J9 x 5 such that the real vector h*(z) - [Z(z)] /* approximates h in the Chebyshev sense. 36In Chapter IV we shall show that in general the optimum solution to this Chebyshev approximation problem yields an error vector having at least (2n+l) components with absolute values equal to the Jl-norm of the error vector. 37The example we have chosen is presented on pp. 79-90 of Ruston's thesis (Ref. 20). However, we shall present this example using the notation developed above. 38It should be mentioned that Ruston intended to find the vector pair - (p**, z*) z x, however, he only succeeded in finding the vector pair (#*, z) e J z x 5. This is why we have stated the approximation problem as shown.

86 Let us begin by writing out the form of Eq. 2. 86 as follows: 6, 1. 0000 0. 4450 0. 2500 0. 4450 0. 2500 0. 1600 0.2500 0. 1600 r0 0.1110 0.1600 0. 1110 + 0.0817 0. 1110 0.0817 r1 0.0625 0. 0817 0.0625 1L o. 0494 _6 0o. 0625 0. 0494 0. 0400 (2. 90) This equation is Eq. 4. 106 of Ruston's report (Ref. 20, p. 78). The vector r' e E2 which yields the minimum value of 11 11 is found using Steifel's algorithm (see Appendix A) to be r 0.2033 r= = (2.91) r 1 -1.0128 and 0.002707 -0. 002707 -0. 000217 6(r') 0. 001801 0. 002318 0. 002707 0.-00267-1 It is noted that (n+l) = 3 components of the vector 6(r') have absolute values which are equal to 116(r')110= 0.0027 (i.e., I61 = l621 = 1661 = 0. 0027) in accordance with Theorem 2.2. Substituting the values of {r.'}, givenbyEq 2.91, intoEq. 2.87, and finding the roots of this polynomial equation yields

87 0. 7369 z =I - 0.2759 Hence, Eq. 2. 88 becomes 1.0000 1. 00000 1.00000 e 0. 4450 0. 73690 0. 27590 0. 2500 0. 53402 0. 07612 0. 1600 0. 40015 0.02100 1 0. 1110 0. 29487 0.00579 0.0817 0. 21729 0.00160 0. 0625 0. 16012 0.00044 0. 0494 0. 11799 0. 00012 0. 0400 0.08695 0.00003 C Solving this equation for f*, which minimizes 11 e_, yields 0. 38427 * I = 0. 60917] 0. 00656 -0. 00424 -0. 00504 -0. 00656 e*(Z) = -0.00584 -0. 00277 0. 00070 0. 00399 0. 00656 Again, it is noted that only (n+l) = 3 components of e*(z) have absolute values equal to lle*(z)ll = 0 00656, i.e., |el = 1641 = 191 - 0. 00656. This concludes the exposition of Ruston's method. However, the following question now arises: Is the point z the stationary point, z*,

88 of the function l e*(z) ]? It will be shown in Chapter IV that, in general, the answer is no, so that ll *(z*)lloc < l*)l. Now, before turning to a detailed consideration of this problem, it should be noted that the comments, which were made at the end of the previous section, concerning the vector [r1, r,..., rn, 1]Tapply also to Ruston's method.

CHAPTER III PRONY'S EXTENDED METHOD The purpose of this chapter is to study "Prony's Extended Method" for solving exponential approximation problems in the q- -spaceo Specifically, we shall consider "Prony's Extended p Method" for solving the following approximation problem: Given a real vector f e Uq, and a set of approximating real vectors {f (,) [Z(z)] (,zz) e F x }, where [Z()] is a q x n matrix, defined by Eq. 1. 18 (Eq. 1. 18a), of rank n(q > 2n). Find the best parameter vector pair (Ip** zp*) e 9z x 5 so that Ilf- [Z(z *)]p **llp * f- [Z(l)]llp (31) The essential part of "Prony's Extended Method" for solving The essential part of "Prony's Extended Method" for solving this approximation problem is that it replaces the parameter vector z by a new real parameter vector r which is related to z by the nh order polynomial equation P(z) = 0, defined by Eq. 2. 65. See Section 2. 3.2 for "Prony's Original Method. " (Chapter II) Since "Prony's Extended Method" simply reformulates the approximation problem defined by Eq. 3. 1, we may generalize the results obtained in this section by using q -norm of e to judge the degree of approximation where p> 1. P 89

90 Let us begin by defining a q x (q- n) matrix [R(r)], of rank (q- n) which depends on an (n+ I)-dimensional vector r, namely,3 r 0... 0 r0;. 0 0' 0 rn' (1 ~... r_ n (3.2) where rr 0 r rr r0 n (3.3) Note that this matrix [R(r)] becomes the matrix [R(r)] defined by Eq. 2.69 when q = 2n. ^ ^ot tatths atix[Ri) bcoms hemari [~i] efnejb

91 The set of all the parameter vectors r is denoted by A, and is defined as follows: Definition 3. 1: The set A, of the parameter vector r, is the set of all nonzero vectors r e E, the n+ 1-dimensional real Euclidean space, so that the roots {z,z2,,...,Zn} of the nth-order polynomial equation n P(z)= = r.z = 0 (3.4) i=0 represent the vector z e 4 Clearly, Definition 3. 1 establishes the relation between the vector z E and the vector r e A. Consequently, if either one of these vectors is known, then the other vector may be found from Eq. 3.4. Let us now establish a lemma which relates the subspaces spanned by the columns of the matrices [Z(z)] and [R(r)]. Lemma 3. 1: Let [Z(z)] be a q x n matrix, defined by Eq. 1. 18, (Eq. 1. 18a), of rank n(q > 2n), and let [R(r)] be a q x (q- n) matrix (defined by Eq. 3. 2) of rank (q- n). Then the n-dimensional subspace of Uq defined by the column space of [Z(z)], C (Z), and the (q- n)-dimensional subspace of Uq defined by n' Note that this relation is not one-to-one, since the vector z e, represents an arbitrary ordering of the roots of the polynomial equation of Eq. 3. 4.

92 the column space of [R(r)], Cqn(R), are orthogonal complement subspaces of Uq; that is, Cn(Z) @ Cq (R) Uq (3. 5) q-n where Cn(Z) I C n(R) if and only if, the components {Zl,'z2.., Zn} of the vector 2L6 n z e g are the roots of the nth-order real polynomial equation n P(z) = r. = 0 (3.6) i=0 where the ordered set {r, r1,... rn define the vector r ^ Proof: Let f [Z(z)] i be a real q-dimensional vector in Cn(Z), ^A where j e ~z, and let g = [R(r)] Z be a real q-dimensional vector in C (R), where y e Eqn. To prove the "if" part of the q-n lemma, we must show that when the vectors z e and r e c are related by Eq. 3. 6, then the inner product (f,) = 0, for all E z and C E Eq-n. Since the inner product between the vector f and g in Uq yields ^ = T T

93 then it suffices to show that the matrix product [R(r)] [Z(z)] = 0, for all z e and r e?. To show this we shall consider the case in which the components of z are distinct. The case in which the components of z are not distinct will be considered separately. (1) When the components of z are distinct, then the matrix [Z(z)] is given by Eq. 1. 18. In carrying out the matrix multiplicaT tion [R] [Z], one obtains P(z1)... P(z ) z P(1) P(z ) 1P(z1) z2P(z) [R] [Z] - lp(zl).zq-n- (z (3.8) 1 n n where n P(zk) = ri Zk k 1,2,..,n (3.9) i=0 Then, since by definition, P(zk) = 0, for all zk e {Zk i k = 1,2,...,n}, the matrix product of Eq. 3. 8 becomes [R]T[Z]= [] (3. 10)

94 (2) When the components of z are not distinct, it can be also T shown that the matrix product [R()] [Z(z)] = [0]. Rather than consideringthe general case, we shall consider the specific —case in which the vector z contains only three identical components. Specifically, let z = z2= z3, and the components {z4z5,...,Zn} are distinct. Furthermore, let us assume that t1 = 0, so that the matrix Z(z) is given by Eq. 1. 18b, where j = 3. Denoting by ai T the ij-th element of the matrix product [R] [Z], then in carrying T out the matrix multiplication [R] [Z], one obtains ail= z P(z1 ai2 = Z At [(i- 1) P(zl) + zlP'(zl)] i i 2 a3 = (At)2 [(() + 2i - 1) z P'(zz) + z P"(z(], a.. =z P(z) = 4, 5,..., n (3.11) i = 1,2,...,q; where P(zk) denotes the polynomial P(z) evaluated at z = zk (see Eq. 3.9), and where The specific case considered here, illustrates the basic relations which are encountered when carrying out the matrix multiplication, [R]T [Z], in the case in which the components of z are not distinct. Although the generalization of this case is straightforward, the notation becomes unmanagable.

95 p,(z) = dP i(i - 1) riz and since6 P'(z) = 0 and P"(z1) = 0, then all the elements given by Eq. 3. 11 must be equal to zero, i. e., a.. = 0, for all i = 1,2,...,q, j = 1,2,...,n. Clearly, this implies that the maT trix product [R] [Z] satisfies Eq. 3. 10. Consequently, we have shown that for all z e g and r e, related by Eq. 3. 6, the inner product of Eq. 3. 7 becomes (f,) = 0 for all P e 9z and y E Eq-n Thus, the subspaces C (Z) and C (R) are orthogonal in Uq. Furthermore, since the dimensionq-n ality of Cn(Z) is n and the dimensionality of Cq_ n(R) is (q-n), then C (Z) and C (R) are orthogonal complement subspaces in U. Hence, n q-n Cn(Z) e Cqn(R)= U where Cn(Z) I Cqn(R) ftq6Note that since z1 represents the third order zero of the polynomial P(z), then it must represent a zero of the polynomials P'(z) and P"(z).

96 In order to establish the "only if" part of the lemma, we must show that if Eq. 3. 5 is true, then (,f) = 0 or alternately, ([z], [R] y) = (3. 12) For Eq. 3. 12 to hold for all 6 e and all Z e Eq-n the matrix T~' z product [R] T[Z], given by Eq. 3. 8, must satisfy [R] [z] = [0] that is, the elements of the matrix [R] T [Z] must be zero. Clearly, this condition will be satisfied only if the set {z, z2,.. Zn } represents the roots of the n-th order polynomial equation given by Eq. 3. 6, that is, n P(z)= riz. = 0 i=0 Thus, the lemma is proved. We have shown that the (q - n)-dimensional subspace, C (R), q-n which depends on the (n+ 1)-dimensional vector r E R, is orthogonal to each approximating vector f. [Z(z)] when (P,z) e Bz x. Recall that this step is similar to that made in "Prony's Original Method, " except that here the dimensionality of the orthogonal

97 complement subspace Cq_ (R) is higher, i. e., (q-n) > n. Let us show that with the aid of the subspace C (R), we can establish an q-n alternate method of solving the approximation problem defined by Eq. 3. 1. This method is the so-called, "Prony's Extended Method." 7 q Let us begin by considering the vector relation in U, that is f = [Z(z)] + (3.13) where f is a prescribed real vector in Uq; [Z(z)] is a q x n matrix, defined by Eq. 1. 18 (Eq. 1. 18a), of rank n(q > 2n); and where both the vector pair (3,z) E z x5, and the real vector e in Uq are unknown. Lemma 3. 1 tells us that there exists a vector [R(r)] I which is orthogonal to the vector [Z(z)] A, when the vector r is in o (i. e., when r is related to the vector z e. via the polynomial equation P(z) = 0, given by Eq. 3.4). The inner product of both sides of Eq. 3. 13 with respect to the vector [R(r)] y yields ([R(r)], f) = ([R(r)], ) (3. 14) because ([R(r)] 7, [Z(zY3 _ ) = 0 from Eq. 3. 12. Since Eq. 3. 14 must be true for all (q- n)-dimensional vectors ~, then'Recall that this equation represents the relationship between the vectors in Uq for the approximation problem of Eq. 3. 1.

98 [R(r)]T f = [R(r)]T e E E (3.15) where f is a prescribed real vector in U, and where r e ~ and E in Uq are unknown real vectors. This equation (viz., Eq. 3. 15) defines the basic relation which must be satisfied when using "Prony's Extended Method" to solve the approximation problem defined by Eq. 3. 1. Note that since all the vectors in Eq. 3. 15 are real, then it suffices to say that f and e are in E. At this point we will denote the vector e given in Eq. 3. 13 by E(J,z) and the vector e given by Eq. 3.15 by e(r). Clearly, these vectors are identical. However, we have chosen this notation to emphasize the parameters on which the vector e depends. Strictly speaking, the vector E, given by Eq. 3. 15, should be denoted by e(~,r) since only the vector z is mapped into the vector r. This point can be best clarified by considering Eq. 3. 15 for the case in which z (i. e., r E I) is initially prescribed in the approximation problem defined by Eq. 3. 1. Recall that the best solution vector o* e z of this approximation problem may be obtained directly from Eq. 3. 13 when e = ep*(z), where the vector 6 *(z) is -P defined by ll ilp = llp lp 7 inlp

99 Let us now show that the vector p*(z) may be also obtained from the vector relation given in Eq. 3. 15. Recall that when the vector z e: is prescribed, the vector r e A is also automatically prescribed. The set of all vectors e (r) in Uq which satisfy Eq. 3. 15 for a prescribed vector r E, i. e., a prescribed matrix [R], is defined as follows: Definition 3.2: For each vector r e A and a real vector f e Uq, the set r denotes the set of all real vectors e(r) in Uq which satisfy the relation T T (3.16) [R(r)] f [R(r)] e (3. 16) Clearly, then, the vector c * which equals the vector e *(z) -p must satisfy min ___^-^ ^ ^P~ g I.)I(3. 17) -p-p e(r) E gr rP This is summarized in the following lemma: Lemma 3.2: For a specified real vector f c U- and a specified vector z c, i. e., a specified vector r E I, the real vectors c *(z) and E *(r) are equal in U, that is, -p -p *(z) = ep*(r) (3.18) where p > 1; if, and only if,

100 (1) the vector p*(z) is defined by p E (Z)l mmin llE *(Z)11p - 4I IFZe()lip (3. 19) where e(,z) is given in Eq. 3. 13, when z e is specified; and, -p (2) the vector e *(r) is defined by E *(r)l A rm aIE() lip -p- p e_(r) E r -- P where E(r) satisfies Eq. 3. 16, when r E c is specified by Definition 3. 1. We have thus shown that if for a specified real vector f E Uq and for a specified r c 9 (i. e., a specified z e 3 ) one obtains the vector Ep*() in 6'r' then, by Lemma 3.2, one can obtain the solution vector fp* e z directly from Eq. 3. 13, by taking e = e *(r). Now that we have discussed the preliminary ideas, we p present the theorem which governs "Prony's Extended Method" for solving the approximation problem stated in Eq. 3. 1... Theorem 3. 1: For each real vector f e Uq, let e ** in Uq denote the vector E(I) which satisfies [R(r)] f = [R(r)] T

101 so that min min min (r)[l where p > 1. Then, the best Pq-approximating real vector in Uq, f ** [Z(z *)] * p -p )] where (p*, zp) e zz x, is given by f ** = f ** -p -p Proof: All of this theorem is contained in Lemma 3. 2, except for the assertion that min min r Ce? Ilep*(r)lip z=e lie i*(z) 1p That this assertion is true should be evident from Definition 3. 1, where the relation between the vector r e W and the vector z e was established. Thus, we have shown that Prony's Extended Method gives an alternate method of solving the approximation problem defined by Eq. 3.1. Such a method leads to the problem of determining the

102 vectors r * e I? and *(r *) e Uq so that -p -P p le*(r *)llp < l(r)ll, p> 1 -p-p p p for all r e e land for all vectors E(r) which satisfy [R(r)] f = [R(r)] (r) It should be noted that the roots of the polynomial equation of Eq. 3. 4 depend only on the direction and not on the magnitude of n+ n+1 r e En. In other words, the vector r e En+ may be normalized with respect to Ir Ip without affecting the roots of Eq. 3. 4 which represent the vector z. Hence, the set of all vectors r e ~ Omay be replaced by the set of all vectors r e En+with 11rl = 1. Since in Chapter IV we shall be interested in the f~ -norm of the vector r; n namely 1 r lI l Z Ir.il, we make the following definition: i=0 Definition 3. 3: The set M of the parameter vector r is the set of all real vectors r E En+l with llrll = 1. Remark: Note that the set P contains all vectors r with the component r = 0. It should be noted, however, that for some prescribed vectors f, if the vector r* is in the set? withr * = 0, then there is another vector r* E c with r * / 0 which yields an n-th order polynomial ~- n containing a root that does not contribute to the approximation. In other words, for some vector r En+E, llrll = 1, with component rn = 0, there is another vector

103 r eE, ll = 1, with component r' 0, that will yield the. same real error vector E *(r) e Uq. This is illustrated in Example 5. 6 -p of Chapter V. Now before illustrating how "Prony's Extended Method" may be applied to solve exponential approximation problems, let us make the following observation about the singular mappings which are dictated by the matrices [Z(z)] and [R(r)]. Recall that Lemma 3. 1 tells us that the vector space U can be decomposed into two orthogonal complementary subspaces, C (Z) and C (R). Therefore, each real vector f in U can be uniquely represented by f = v(z) + w(r) (3.20) where v(z) and w(r) represent the orthogonal projections of f onto the subspaces C (Z) and C n(R), respectively. Let us now show how "Prony's Extended Method" simplifies the least squares exponential approximation in the U space; i. e., the 8 case in which p = 2 in Eq. 3. 1. As is well-known, 8for each fixed z in g this approximation yields an error vector, *(z), which is orthogonal to the vector f*(z). Hence, e*(z) represents the 8See Theorem 2. 1. (Chapter II)

104 orthogonal projection of the prescribed f onto the orthogonal complement subspace of C (Z) in Uq. Clearly then, in terms of Lemma 3. 1, e*(z) lies in C_ (R). Hence, the vectors f*(z) and q-n e*(z) have the relation of the vectors v(z) and w(r), respectively, of Eq. 3.20. Therefore, we may represent the error vector, *, as a function of either the parameter vector z or the parameter vector r; that is, e*(z) - [Z(z)]* (3. 21) where * [ZTZ]-1 TZf (3.22) or, alternatively, *(r) = [R(r)] y* (3.23) where * = [RTR-1RTf (3.24) Thus, to find the vector z* in g which minimizes lel112, we 9 can either (1) determine the stationary point z* of the function g a 11e(~)112 That is, the point where a) = O forall k= 1,2,..,n.

105 l_~*(z)ll2, where c*(z) is defined by Eq. 3. 21; or, (2) use "Prony's Extended Method" to first determine the stationary point r* in E of the function Il *(r)ll2, in the set II rl = 1, where e*(r) is defined by Eq. 3. 23, and then find the n zeros of the polynomial equation n P(z) = r. z i=0 In summary, we have shown that "Prony's Extended Method" gives an alternate approach to the solution of the exponential approximation problem. At this point, it should be mentioned again that to carry out this approximation method, it is necessary that the prescribed values of {f(ti)} be obtained at equal intervals of t.. In the next chapter, we shall use "Prony's Extended Method" to solve the Chebyshev exponential approximation problem in the U3 space, i. e., the case in which p = oc in Eq. 3. 1.

CHAPTER IV OPTIMAL CHEBYSHEV APPROXIMATIONS AT DISCRETE POINTS IN THE TIME DOMAIN 4.1 Introduction The purpose of this chapter is to study the Chebyshev approximation problem stated in Section 1. 5 of Chapter I, i. e., the Chebyshev approximation problem in a finite dimensional space occurring when the approximating subspace is not fully prescribed. Specifically, we shall consider the following approximation problem: Given a real vector f in U, and a q x n matrix [Z(z)], defined by Eq. 1. 18 (Eq. 1. 18a), of rank n(q > 2n); find the best Chebyshev parameter vector pair (j**,z*) e z x 5, such that if f** [Z(z*)] **, (4.1) then, llEl**l ll- f** llo i < [zll (z)] illo (4.2) for all (,z) e z x 5. In this chapter we shall be primarily concerned with the existence and the properties of the best Chebyshev parameter vector pair (J**,z*) e z x 5. Most of the results will be presented 106

107 in terms of "Prony's Extended Method" which was developed in Chapter III. In Section 4. 2, we shall offer some intuitive notions relating to the character of the resulting best Chebyshev error vector e ** U In Section 4. 3, we shall give the existence theorem and the bounds within which the value of II e ** Io must lie. The properties of the best Chebyshev vector pair (J**,z*) e z x will be given in Section 4. 4. Here, we shall show that the vector pair (**, z*) e sz x a represents the vector pair which defines the best Chebyshev approximating vector in some (2n+ 1)-dimensional reference subspace U2n+ of U, where c {w:w = 1,2,.., (2 n In Section 4. 5, we shall be concerned with finding the best Chebyshev approximation in each (2n+ 1)-dimensional reference subspace U2n+ of U. w Specifically, we shall give the theory which renders the computational method presented in Chapter V. Although we are not primarily concerned with the uniqueness property of the best Chebyshev approximation in this chapter, a discussion of this subject is presented in Section 4. 6. 4. 2 General Approach to the Solution of this Chebyshev. Approximation Problem Before studying in detail the approximation problem considered -here, we shall offer an intuitive notion relating to the character of the resulting Chebyshev error vector e** e U.

108 Specifically, if we consider Theorem 2. 2 and note that our unknown parameters lie in a 2n-dimensional parameter space Jz x we should expect the absolute values of (2n + 1) components of e** to be equal to 1 e** 1o. Admittedly, the expansion of the parameter space does not occur through the doubling of the dimensionality of the approximating subspace, but rather occurs through freeing the orientation of the approximating subspace, Cn(Z). The achievement of at least (2n + 1) error components of equal magnitude is not a surprising result since when q = 2n, the error components are usually equal to zero. As a further digression, the approach to the solution of this Chebyshev approximation problem is now outlined. We begin by seeking the solution of the Chebyshev approximation problem for some fixed (i.e., prescribed) vector z e. This problem is identical to the problem considered in Theorem 2. 2, since now the matrix [Z] is prescribed. From Theorem 2. 2 we find that the best Chebyshev approximation vector f* - [Z] a*, for a prescribed vector z e, yields an error vector, e* = f- [Z] *, which will have at least (n+ 1) components equal in absolute value to Ie_* l. It is evident that e" depends on both the prescribed real vector f in U4 and on the prescribed q x n matrix [Z] (in other words, on See "Prony's Original Method" which is presented in Section 2. 3. 2, Chapter II.

109 the vector z e a ). The fact that e* is a function of the vector z will be emphasized by representing e* by e*(z). If now the vector z e is not prescribed, then we claim that by varying z, we can decrease the value of IIE*(z)II so that the absolute values of at least (2n+ 1) components of E*(z) are equal to Ill*(z)ll1o. This claim is based on the fact that we have acquired extra n-degrees of freedom, corresponding to the n-components of the parameter vector z, with which to modify *(z). The resulting vector e*(z) is denoted by ** = e*(z*), where z* denotes the z which gives the minimal value of lle*(z)l. It should be mentioned that if some of the components of z* are equal to zero, then the above mentioned character of the resulting error vector, E**, may not necessarily hold, i. e., the vector e** may consist of less than (2n+ 1) components equal in absolute value to I e** lio This will be discussed further in Section 4. 5. 4. 3 The Existence of the Best Chebyshev Approximation We begin by proving that every prescribed real vector f e Uq has a best Chebyshev approximating vector, f**, defined by Eq. 4. 1. Recall from Theorem 2.2 that for each z e there exists a E* e z such that This claim rests on continuity considerations, see Section 4.3. 3The other n-degrees of freedom correspond to the n-components of the parameter vector j.

110 ll *(z)l11 If- [Z(z)] *ll <_ Ilf- [Z(z)] 1 (4. 3) for all 3 e,z. Hence, to show that there exists a best approximating vector, f** = [Z(z*)] **, to a prescribed real vector f e U it suffices to show that there exists a z* c 5 such that ll )(z*)1II, L_ l*(z)11l (4. 4) for all z e 5. Since our existence theorem is based on the fact that the function 111*(z)l1c is a continuous function of z e, let us first establish the following lemma: Lemma 4. 1: Let the vector e*(z) e U be a function of z e, defined by e (Z) = f- [z(z)] * (4. 5) where f is a prescribed real vector in U; [Z(z)] is a q x n matrix, defined byEq. 1. 18 (Eq. 1. 18a), of rank n(q > 2n); and where (* is a vector in Bz such that IL*(z)l11 f - [Z(z)] * K < f- [Z(z)] ll(4. 6) for all (3 e 4. Then the function Ic *(z)l1o is a continuous function of z cc function of z e.

Ill Proof: To show that lle*(z)l{{ is a continuous function of z c, we must show that 1lE *(z)l{ { is continuous at each point z e, that is, for every real number 71 > 0, there exists a real number 4 6 > 0, such that z e and lL-z olIl < (4.7) implies that 1 ll*(lZ)}l~(l - ll*(Zo)lloc1l K (4.8) Recall that e, =;=: e.3*,z,) and that the vector * e Bz is a function of z e 7. Hence, let us denote the vector e*(z) by E(*(z),z). Consider the relation between the function ll(_0,z)10) at the four points in the e1z x space shown in Fig. 1, p. 112. It can be shown (see Appendix B) that for z and z e: (1) if z / 0 and if li -Zolll < - iZollc, then -'O -- 2ii.~(~*(o),)li - 1~(.,(:o):..o)1[ <' ()!1 z -o (4 9) ~ r:: -.... -'. Z 4It should be noted that for each z and z e, Eq. 4. 7 defines an open set in 5, since Eq. 4.,7 r;presents the intersection of the subspace 5 of Un with an open pphere in Un (see Ref. 21, p. 64).

112 (:*(z), z) ( z*(Z), z) P*(z).-.-.-. -x /*(z) - - — x (*(z), z) ((z), z z z 0 Fig. 5. The points in the wz x g space considered in Lemma 4. 1.

113 where Cl(Zo) = 211f lo_ (J) (4.10) and I e(i*(z), z) - ll *(z) zo) 1 < C2(Zo) ll - oll (4.11) where -1 q-1 c2(zo) = 2(n+ 1) Ilf l, 11l zo ( 11 (4. 12) j=x (2) If z = 0, and if 1Iz.11 < 1, then ll(_(Z*(o), )ll - llI(*(Zo)Zo)lloo < 2I f ll1c Lz (4. 13) and llL((z)z)ll -le(*(z)o)lloo) 2 llf ll llj1 (4. 14) Hence, it can be seen from Eqs. 4. 9 through 4. 14, that for a given t1/2 > 0, there exists a positive constant defined by min{2_' I2' Zo, when z 0 2c 2o 0 6 = min{41fl 1}, when z = (4.15) such that f < suchthatif 112 - zo[1U < 51, then

114 IIlr(o*(-)0IC 11o(I*_ (_ I1 (4 2. 16) and there exists a positive constant 62, defined by mi22( )' 2 when z 0 min 2~c ~^, -T II 211 >, - when z 2 0 2 J-zo)J 2 mint ~4 f,1, 1when z = 0 (4.17) such that if z - zo l < 62, then ill (*(_,z))llo - 11e (*(Z),Z_)10 < j 2(4 18) Recall from Theorem 2. 2 that for a specified z e we have the relation E111 e(,Z)l > 1 e.(, Z)llo for all e z. Hence, we note that 11_(*(Z )z)I - ll(1)*(z),z)ll o dd > 0 (4.19) and le(L*(z),Zo)Io - ll ( o) loo = d2 > 0 (4.20)`Adding Eqs. 4. 19 and 4. 20, yields

115 dl +d2 = {le *(z) z)ll0 - i(*(z)z)l }0 + {lle ('*(z), z)iO - lEj (*(zO Z) l ix} < IE 1i(z), zz) - lI( (z)), zI + (zo)) - 1 (00) 00 (4 21) If IIz - z II < 6 where 6 ~ min5{6 1 }, then the inequalities of Eqs. 4. 16 and 4. 18 are satisfied and the inequality of Eq. 4. 21 yields d + d <d2 +2 2 1 - (4. 22) 1 2 2=... Let us now substitute the value of E111(*(z), 011o, given by Eq. 4. 20, into Eq. 4. 18. This yields | lle(*(Z), Z) *, OO ( Z'(3 zll d < 2 -((^) - o 2 < or alternately (i*(z, z))l0 - I(*(Z)o) O + d 2 (4.23) -o 0 0. 00 Similarly, substituting the value of ll(*(z),z)lt, givenby Eq. 4. 19 into 4. 16, yields 1e(*(z),_z) - lie(2*(z:):)o < 2 + dl (4.24) Adding Eqs. 4.23 and 4. 24 and using the relation of Eq. 4. 24 yields

116 1ie(Q*(),z)l) e- (Il(oo (100 < 2 ( + d + d2) < (4.25) Hence, given an 1 > 0, we can find a 6 > 0, which is given by 6 =min{ 6,62} (4.26) such that if z- zl1 < 6, then from Eq. 4. 25 we have le *(z) ll - l *(Zo)ll < 77 (4.27) Thus, the lemma is proved. Theorem 4. 1 (Existence Theorem): For each real vector f e U, there exists a vector r* e, where 9 is defined by Definition 3. 3, such that tle *ll o e*(r*) 1 0 < l * *(r)1 (4. 28) for all r e t and real vector e* (r) Uq which satisfy the relation [R(r)] f = [R(r)] * (r) (4.29) where [R(r)] is the q x (q - n) matrix defined by Eq. 3. 2. Proof: Since the set of vectors r E (i. e., r E E+ with Hill 1 = 1) forms a compact set, then if the function 1 e* (r) l is a continuous function of, it attains its minimum on this a continuous function of r e ~94, it attains its minimum on this

117 5 set, i. e., there exists an r* such that Eq. 4. 28 is satisfied. Hence it suffices to show that the function IIe* (r) 11 is a; continuous function of r E6.Since by Lemma 3. 2 the vector e *(r) = e*(z), and since by Lemma 4. 1 the function lie *(z).,l is a continuous function of z e, then the function lE *(r) IIis a continuous function of r e A. Recall from Definition 3. 1 that for each vector r.e A, the vector z e g can be obtained from the n-th order poly-, nomial equation n P(z)= E r. z = 0 (4.30) i=O Furthermore, if r 0, the vector r may be normalized with n 7. respect to Ilrll, as long as Ifrll r 0, without affecting the -P p values of the roots of Eq. 4. 30 which represent the vector z. In other words, for all r e 9 for which ilrllp 0, the value of lle *(r)I is independent of lr 11. Furthermore, it can be shown 00 p that the continuity of the function le *(r) I can be extended to all 5See Ref. 21, Chapter 4. Since in this chapter we shall consider only the case when p = 0o, we shall drop the subscript p in E * -p 7Note that IIrll =0 occurs only if r =0, a point in which is excluded by assumption, since r = 0 implies that the polynomial P(z) has no roots, or z.

118 r E R. Thus the theorem is proved. It should be noted that for some vectors r* e? there exists 8 no vector z e. For example, the case in which the minimal value of tl *(r)ll occurs when the vector r* e t has the com00 ponent r* = 0, and there exists no other vector r* E 9 which *n *0 9 yields the same value of II *(r)II when the component rn # 0. 00 n This is illustrated in the following example: Example: Given q = 5, n = 2, and f= 5 Since n =2, then the vector r = [ rr r 2]T Let us assume that the initial approximation yields the following vectors r z, and E* (r) 0. 5 0.5 1 [ ] -:+1. 732 0 r 0 z=, *(r) = 0 -1. 772 -0. 333 -77 -0. 5 0. 5 8In other words, some roots of Eq. 4.30 are at infinity. See page 102.

119 If ell*(r)lO = 0. 3, then the vectors r, z, and e* are 0.3 1-0. 033 1. 227 r.0.71 6, z -0, 3 -1o. 347 0. 0785 -10. 3 0.3,4 If 8e*(r)11 = 0. 1, then the vectors r, z, and e*(r) are 0.1 1. 000 0. 0068 1.089 r= f.-0875 0z= -* = -0. 1 32.230 -0. 028 The minimal value of Ie *(r)ll = 0, which occurs when r* = -1, z* = _0, L_ Two points should be noted concerning the above example. First, although the example illustrates the nonexistence of a finite z for the best Chebyshev approximation, we can obtain a finite z for some prescribed neighborhood of fIe**l 1, The second point to be noted is that the prescribed vector, f, represents the sample values of a nonrealizable impulse response function which is not generally encountered in network synthesis. - 10This point will be discussed further in Chapter VI. This point will be discussed further in Chapter VI.

120 Let us now define the vector r* e 9 which yields the minimum value of e *(r) as follows: Definition 4. 1: The vector r* is said to be the best Chebyshev solution vector in r, if 11e*(r*)11 < lfe*(r)lI (4. 31) for all r e 4 which satisfy the relation LR(r)] f= [R(r)] *(r) (4.32) where f is a prescribed real vector in U, [R(r)] is a q x (q - n) matrix defined by Eq. 3.2, and e* (r) is defined by Eq. 3. 17. Some further examination of Eq. 4. 32 reveals that one can find the lower bound which the value of tie*(r.*)BO can attain. First, we note that the left hand side of Eq. 4. 32 may be written as [R(r)] f=[F]r (4. 33) where f 1 f fl f2 fn+1 f2 f3 fn+2 [F] = | * = (q - n) x (n + 1) matrix) (4. 34) f f f q-n q-n+l'' q "1Recall from Lemma 3. 2 that e*(r) = e* (z).

121 Let us substitute Eq. 4. 33 into Eq. 4. 32 and take the Q -norm of 00 both sides. This yields, II[F] rll= II[R(r)] 1T * n n max m * ax l<i<q-n jIZ0 rj i+j l<i<q-n j0 j i+j (n)( < I rI ) max IE*i (4t 35) -( j=0 i ) (l<i+j< q i+j - ) -!t. 5) Hence, from Eq. 4. 35 one obtains the following relation: r le*(r)lII > Il[F] 13 11 (4. 36) for all r E A. The matrix [F] is a prescribed real matrix, and the n+1 function U [F] rjj_ is a continuous function of r E E. Hence, if we allow r to vary on the unit sphere defined by \ri\ I = 1, then the function 11 [F] rII attains its minimum at some point on that sphere. Let us denote this minimum value by n. Thus, from Eqs. 4. 31 and 4. 36, we note that, for each12 r e, the value of le *(r*)lj lies in the interval defined by lle*(r)llo _ l el*(r*)l loc (4. 37) where rmin 71 =llI=t 1 II[F] rlI (4. 38) Note that again we assume that the minimum will not be attined at the point where the component rn = 0 (see Chapter III).

122 Remark: Note that if ij = 0, then the (n+l) colums of IF] are linearly dependent. This implies the existence of an exact solution, i. e., there exists an r* c A, such that e *(r*) = 0. Clearly, this represents the case in which the interpolation of "Pronygs Original Method" (Section 2. 3. 2, Chapter II) may be extended to q-points, where q > 2n. This case is illustrated in Example 5. 6 of Chapter V. The following lemma summarizes the above results: Lemma 4.2: Let r* be the best Chebyshev solution vector in ~ and let *(r*) be the corresponding real error vector in U. Furthermore, let r be the vector in? which yields the value of r defined by Eq. 4. 38, that is r r. l[F] [F] (4. 39) llrII1 Irllli for all r e 9, and where [F] is a prescribed (q- n) x (n+ 1) matrix defined by Eq. 4. 34. Then the value of lle*(r*)llj lies in the interval Iie*(_r)li ~ Ile*(r*)110 > (4.40) and the equality holds, if and only if, any (n+ 1) consecutive components of the vector e *(r) satisfy

123 (i) leM+j ( r) = l(e*(r)ll, for j=0,l,...,n and (4.41) (ii) either sgn e* (r) = sgni, r for all j =0,,..., n, or sgn ej (r)= -sgnr, for all j = 0,,...,n (4.42) M+j where M e {i= 1,2,...,q-n}. Proof: All of this lemma, except for the conditions of Eqs. 4. 41 and 4. 42, has been obtained in the discussion above. To show that the conditions given by Eqs. 4. 41 and 4. 42 are needed to make the inequality signs of Eq. 4. 40 become equality signs, let us examine Eqs. 4. 35 and 4. 36. Clearly, if the condition given by Eq. 4. 41 and 4. 42 are satisfied, then the inequality sign of Eq. 4. 36 becomes an equality sign. To prove the "only if" part, let us suppose that there exists a vector r' e? such that l~*(l')I oc = Then we can write

124 e.i*(r') = ~i 7, i= 1,2,...,q where -1 < C. < 1. Substituting into Eq. 4.35, we obtain [F] r' 11 =11 [R(r)] e )ll n = (1<i<_q-n Ij J i +j However, since R = IL lt 1 [F] r'tl0 then we obtain n ilr 11 = l<i<q-n rj ij Denoting by M, the i for which the right hand side of the above expression attains its maximum, we have n 11r} = I Zrj M+j j=0 which can hold only if sgn rj 1,. 0 n M+j or M+j -sgn r' j=0 1,... Thus, we have le +j(r') I = ) = = ll*(r')llc and the condition of Eq. 4. 42. Hence, the lemma is proved.

125 13 The following example illustrates Lemma 4. 2: Example 4. 1: Given q = 5, n = 2, and 3.9 3.1 f = 1.9 1. -0. 1 Since n = 2, then the vector r is given by r r = r r2 and from Eq. 4. 34, the matrix [F] is given by 3.9 3.1 1.9 [F] = 3. 1.9 1.1 1.9 1.1 -0. 1 The vector r, which gives the minimum value of l [F] rll, is found to be 13This example is considered in detail in Chapter V, Example 5. 4.

126 1 r -2 1 and the corresponding vector [F] r is -0.4 [F] r = 0.4 -0. 4 Therefore, the minimal value of I [F]rli in the set \\r\1 = 1, i.e., the value of q; given by Eq. 4. 39, is A I[F] 10o 0. 4 77: ~ 0. 1 = 4 l1rlll To find the best Chebyshev error vector e*(r), we must first obtain the vector z from Eq. 3. 4, namely from P(z) = - 2z + 1 = 0 This yields Since the components of z are not distinct, then, if we assume that the initial sample of f(t) (i. e., the value represented by the component f1)

127 is taken at t = 0, the matrix [Z(z)] is given by4 Eq. 1. 18c, namely, 1 0 1 1 [Z()] = 1 2 1 3 1 4 Consider the vector relation 3.9 1 0 1 e 3.1 1 1 2 + 1.9 = 1 2 L + 11.1 3 -0.1 1 4 E The vector f* which minimizes the value of 11 Io is found to be 4 * = and the corresponding best Chebyshev error vector is The reader is referred to Example 5. 4 where we have used the < matrix [Z(z)] to be given by Eq. 1. 18. The results of the minimizing procedure are identical to those presented here.

128 0. 1 0. 1 e _*(r) = -0.1 0. 1 Since the value of lle*(r)l1 = 0. 1 = 0, then the vector r, found above, is the best Chebyshev solution vector r*. Furthermore, it is noted that the sign configuration of the vector e*(r) satisfies Eq. 4.42 of Lemma 4. 2. It should be noted that the vector r defined in Lemma 4. 2 may be obtained by extending Ruston's method (see Section 2. 3. 3.2, n+1 Chapter II) to the case when the vector r e E is restricted to the set defined by i r l = 1. Recall that Ruston's method yields the vector r e E n+for which the function5 1 [F] rll is minimum for n+1 16 all r e E, when r = 1. It suffices to say that if some other n component of r e E is assumed to be equal to one, then the n+1 function i [F] rl1i may attain its minimum at some other r e E Let us denote by r(, the vector r c E+ with the j-th component equal to one, and denote by r ), the vector r for which the function I [F] r()ll is minimum. Since there are (n+ 1) distinct vectors Note that the vector [F] r represents the vector 6(r) defined by Eq. 2.93. 16See the discussion concerning Eq. 2. 93 and Eq. 2. 94.

129 _(j) En+1 (j) r(j e E, then we have a set of (n+ 1) distinct vectors r( namely, {() j =,...,n+1} and a corresponding set of values { [F] r()o: j = 1,..., n+1}. Hence ij, defined by Eq. 4.38, is min II F]l r II 77 - 1Kj~nn+1 (4. 43) 1<j_ n+l (J) and the vector r defined by Lemma 4. 2 is the vector r in the set { ): j=,...,n+ 1} which yields 1. This may be best illustrated 17 by the following example: Example 4.2: Given q = 5, n = 2, and 0 2. 8 f 2 1 1 Hence, 0 2.8 2 [F] = 2.8 2 1 2 1 1 17 This example is considered in detail in Chapter V, Example 5. 3. -~~

130 and r0 r = r1 r2 Let us denote the vector [F] r by the vector 6(r), i. e., 6(r) = [F]r. The set of vectors r(j):j = 1,2,3} which minimize the set of functions {, (r(j)I:j = 1,2,3} is given in Table 4.1. From Eq. 4.43 j 1 2 3 [r ~(j) 1.0 -0. 1346 -0. 04098 r J -0.8000 1.0 -0. 60109 r1 (J) 00.3467 -1.17691.0 6 1(j) -1.5467 0. 4462 0.31694 6() 1. 5467 0.4462 -0.31694 2 63j 1. 5467 -0. 4462 0. 31694 116 11 ()~ <0. 7205 0. 1930149 0. 1930116 Table 4. 1. Results of Example 4. 2. 9

131 and the set of values (: j = 1, 2,3 where ( [F] r( we obtain that 1 = 0. 1930116. Furthermore, vector r represents the vector r defined by Lemma 4. 2. The following two points should be noted concerning the above results: First, if the vector r obtained by Ruston's methods yields 77, namely if r(n+l) = r, then, byEq. 4.40, Ruston's final Q-norm of the error vector, e *(r)lI, represents the upper bound of the interval within which the optimum Qq-norm of the error vector, 00 Il *(r*)ll0_ lies. Furthermore, if the interval [71, le*(r)100 ] is small, then for a specific application, where only the final error of approximation is of interest, Ruston's solution vector r e c may be quite satisfactory (see Example 5. 3). The second point to be noted is that if 71 = 0, then the vector r = r* and the best Chebyshev approximation yields lie*(r*) 10 = 0 (see Examples 5. 5 and 5. 6). 4. 4 Properties of the Best Chebyshev Approximation In this section we shall study the properties of the best Chebyshev approximation given by Eqs. 4. 1 and 4. 2. Recall, from Section 19 2. 3. 1.2, that when the vector z e $ is initially prescribed (i. e., the q x n matrix [Z(z)] is fully prescribed), then the vector * e, z Note that the vectors r )/jIr l and r /lrl ll are almost identical. 19See Corollary 2. 1.

132 which defines the best Chebyshev approximating vector, f*, in U, will also define the best Chebyshev approximating vector in some (n+ 1)-dimensional reference subspace of Uq taken from the set n+ 1 {U: = 1, 2,... )} In this section, we shall show that V n+~ when the vector z e 5 is not initially prescribed, then the vector pair (f3**,z*) E Z x 0, which defines the best Chebyshev approximating real vector in Uq, will also define the best Chebyshev approximating vector in some (2n+ 1)-dimensional reference subspace20 of U, taken from the set {Un+: w = 1,2,..., However, - w' "''' 2n+l before turning to a detailed consideration of this property, we must first define the set of distinct (2n+ 1)-dimensional reference subspaces of Uq given by {U2n+1 w = 1,,..., and establish W ~ ~'''~ (2n+l the relationship between the prescribed and approximating vectors in each (2n+ 1)-dimensional reference subspace of U. Let us define the (2n+ 1)-dimensional reference subspace, U2, of Uq according to Definition 2. 1. Clearly, when m = 2n+ 1 in Definition 2. 1, one can form a set of (2n) distinct (2n+ )-dimensional reference subspaces from Uq, which will be denoted by the set {U2n+l w = 1,2,..., ( )}. In each (2n+ 1)-dimensional referw 2'''2n++ ence subspace Uw+;w =1,2,..., ( 2n+), the (2n+ 1)-dimensional w;w 12n+1 projections of the vectors f, [Z(z)], and e in Uq, are given by 20The increase in the dimensionality of the reference subspace is based on the fact that the parameter vector z e ~ introduces at most n new degrees of freedom.

133 f() = [I]T f U2n+l (4.44) ~ ^ w^~ w [z((z)] =I []Z(T ] l Uwn (4.45) and E(w) [I T e 2n+1 (4.46) where [Iw] is a q x (2n+l) matrix defined by Eq. 2. 26. Therefore, the basic relation between the prescribed and approximating vectors in Uq, namely, f = [Z()] + e_(,z) (4.47) implies the following relation between the prescribed and approximating vectors in each (2n+l)-dimensional reference subspace, 2n+1 U w f(w) [z((z] (w)(,z) (4.48) where w = 1,2,...,(2n+1) The Chebyshev approximation problem in each (2n+l)-dimensional reference subspace 2k, k e {w = 1,2,.., (2 )} is sdi r(k) 2n+1l nl stated as follows: Given a real vector f() e U and a real apprxiainvcor(k) dfndbq4 sk proximating vector [Z k(z)] i defined by Eq. 4. 45; find the best

134 21 Chebyshev k-th reference parameter vector pair2 (ik**,'k*) x, such that if f(k) ( A[Z (Zk,)][ = (k)(zk* (4.49) then {le(k) *k*ll ^ A11 f(k) — (k)4 *)110 < f(k) (k) (4.50) for all (,,z), x It should be noted that the approximating vector l(k**, k*) e U corresponding to the vector pair (k*,zk*) e Bz x might not necessarily define the best Chebyshev approximating vector in Uq, for the given vector Zk* e 2, since there might exist a B* k** e Z * such that ll_(S*'Sk*)lloc < ll~k'k^OG llo ^(4 51) where e (* -k*) and e (k** Zk*) in U are defined by E( *,k*) = f - f(*,z[Z = Z )] and ancl- - - I I I e(_ " ) f f(k Z) ) f *[Zzk*)] 3 k * 21For clarity, we shall refer to the best Chebyshev paramet~er vector pair which defines the best Chebyshev approximation in Uk, as the "best Chebyshev k-th reference parameter vector pair," and denote it by (k**,Zk*).

135 The following theorem establishes the relation between the best Chebyshev parameter vector pair (i**,z*) and the best Chebyshev k-th reference parameter vector pair (k**,Zk*): Theorem 4. 2: If there exists a parameter vector pair (il',z') cz xg 2n+1 and a (2n+l)-dimensional reference subspace U, where nq+thtte 2n+1) c e {w = 1,2,..., (2n+1) such that the 2n+1-norm of the error vector e(C) (,z') e U2n+l satisfies (c) (C) lle (c)(, z )ll<o IIlE _ (_ Z) (4. 52) for all (i,z) e B xg; and if the &q-norm of the error vec~- Z o00 tor e(',z') e U satisfies I1 e(i l',lz') = Ie (c) (', z' then the vector pair (I',z') is the best Chebyshev parameter vector pair in RZ x, i.e., (, Z') = ** Z*) Proof: First we note that for each parameter vector pair E eV (:Z) E fz xg }( )o = max {li(w)(ii,z) oo}(4 53) 1<w < (2n+1) or alternately,

136 Il(,z)11, > llle(w)( Z)ll, w = 1,2,..., (2n+1) (4. 54) where E(W)(,z) represents the projection of the error vector q onto U2n+1 q. 2n+1 (i, z) e U ono Un+t of Uq. Let us denote by Ucn+ the (2n+1)2n+1 dimensional reference subspace for which the 2 -norm of the c0 error vector E(C),z) E U2n+1 when (,z) = (',z'), satisfies (c) = 1, m ) max l<w' (z )) 2n+1 Hence, from Eq. 4. 53 we have (cllel)( _)l =} ll(,2',Z' (4.55) If I(C)(',z')I(,_ satisfies Eq. 4. 52, then from Eq. 4.55 we obtain \le__('z')-l lle(c)(.',z')!c <_ 91c)(X z)\\ (4. 56) for all (3,z) e z x. Moreover, by using the relation of Eq. 4. 54, we have 1 E(c)( Z)l < 11 (,z)ll1 Then the relation of Eq. 4. 56 yields Illle( z)l9, < }le_(,z)llc (4 57) for all (i,z) e'z x 5. However, since by definition the best Chebyshev parameter vector (**, z*) e Z x 5 yields an error

137 vector (E**, z*) U with a o-norm satisfying l}e(**,*) II_ Il( < eZ)ll for all (I,z) e 6 x, then from Eq. 4. 57 we note that (,' z') = (**,z*). Thus, the theorem is proved. Remark: Note that since the vector pair (i,z), considered in Theorem 4.2, must satisfy Eq. 4. 52, then (t,zt) represent the best Chebyshev k-th reference vector pair (k**,zk*) defined by Eq. 4. 50, when k = c. It is convenient to formulate the above results in terms of "Prony's Extended Method." It can be shown (see Appendix C) that, in terms of "Prony's Extended Method, " the relation of Eq. 4. 48, i. e., the relation between the vectors in each (2n+l)-dimensional reference subspace U n w = 1 (2q1) is given by ""'(,)l~f~w [rl, w 12nl+1 [A(W)( )]Tfw)- [A(W)(r)]TE(W) (4.58) (W) (W) [A )_ - _ -[AW)]e, w = 1,..., (2n+1) (4.58) where (w) and e are given by Eqs. 4. 44 and 4. 46, respectively; andwhere2(w)2 and where [A((r)] is a (2n+l) x (n+l) matrix defined by2 See Appendix C, Definition C. 3, when m = 2n+1.

138 (w) ^ (1i@) 0...0 ~ 2, 4 (r) X(W) 3, 2_0 (W) (r) (W) n+-, 1 n+ 1, n+ l [A^w)(r)] (4.59) (W) (w) 0 X- (r) X (r) n+2, 2 -n+2,n+ 10 0 o 0 0w) (r) 2n+ l, n+lr where the nonzero elements, X.(W)(r), of the matrix [A(w) (r)], represent some polynomial in the n+l-variables {r0, r,..., r } denoting the components of the vector r e 6. The following three points should be noted concerning the matrix [A(w)(r)]: First, the (n+l)-column vectors of [A(w)(r)] span an (n+l)(w) 2n+1 dimensional subspace, C (Aw), in U which is orthogonaln+1 w 23 (w) complement to the n-dimensional subspace, C (Z), spanned by the n column vectors of the (2n+l) x n matrix [Z(W)(z)]. Second, 23Note from Eq. 4.45, that since the matrix [Z()(z)] = [Iw]T [Z(z)], and since the matrix [Z(z)], defined by Eq. 1. 18 (Eq. 1. 18a), is of maximal rank, n, for all z e, then the matrix [Z(W)(z)] is of maximal rank, n.

139 since the matrix [A ()(r)] is a function r e, then the relation between the (q x n) matrix [Z(z)] and the q x (q - n) matrix [R(r)], given by Lemma 3. 1, is preserved. Third, for a given pair of vectors r and (k) 2n+1 tors r c and ek e Uk, which satisfy Eq. 4. 58 for some k {w = 1,..., (q+1) one can obtain24 a unique vector e c Uq which satisfies the relation [R(r)]T f = [R(r)]. Let us now define the best Chebyshev k-th reference solution vector r * e as follows: Definition 4. 2: The vector r* is said to be the best Chebyshev k-th reference solution vector in A, if (k)f a 2n+1 for all r e f and (k(r) e U k which satisfy the relation [A(k)(r)]Tf(k)= [ (k) =T E(k)(r)] (4. 61) where f(k) and (k)(r) represent the projection of f and e(r) e Uq onto the (2n+l)-dimensional reference subspace U n k e {w = 1,2,..., (o )}, according to Definition 2. 1; and where [Ak)(r)] is a (2n+1) x (n+1) matrix defined according to Definition C. 3, when m = 2n + 1. 24 See Appendix C, Theorem C. I, when m = 2n + 1.

140 Remark: Note that the vector e*(k)(r defined by Eq. 4. 60, (k) and the vector e ()(k**,z), defined by Eq. 4.50, are identical, since the vectors r * and z * are re-k -k (k) lated by Eq. 3.4. Hence, the vector *(k)(r *) may 25 (k) _ be denoted by e (k)(*,r). This notation is particularly helpful when one wants to differentiate between the following two error vectors in Uq: (1) the vector e(k**, r) which denotes the vector e e Uq obtained from a given value of rk* e and _(kr) ek U'r, by using Theorem C. 1; and (2) the vector e*(rk*) which denotes the vector e e Uq obtained for a given value of r * e and with a f q-norm satisfying 00 l1 e *(rk*Q )llo *I*c< Erk*)lloc for all e z. Clearly, the vector e*(rk*) e U represents the vector e(B*,zk*) discussed in Eq. 4. 51. 25 k Note that our notation does not distinguish between the vector *(k)(rk*) defined by Eq. 4.60 and the projection of the vector e*(rk*) e Uq onto 2 n+ 1 *(k) Uk. Through this thesis we shall use e( (rk*) to denote the vector defined by Eq. 4.60. If the projection of the vector e*(rk*) U2n+ i considered and it does not satisfy Eq. 4. 60, then for clarity we shall use the notation e (k)(*, r*)

141 The following theorem establishes the relation between the best Chebyshev solution vector r* e s, defined by Definition 4. 1, and the best Chebyshev k-th reference solution vector r * e, defined by Definition 4. 2: Theorem 4. 3: Let r * e t be the best Chebyshev c-th reference c solution vector in the (2n+1)-dimensional reference subspace 2n+n q )}, U^ +, where c {w = 1,2,..., (2n+1 ) namely le *(C(rc)llo < e c)(r)\) (4. 62) for all r e and (C) Ucn1 satisfying Eq. 4. 61. If the occc!q-norm of the error vector e*(r *) e Uq satisfies OC ~"c le*(rc*)llo = 116* (r ) (4.63) then r* = r C - where r* e? is the best Chebyshev solution vector, defined by Definition 4. 1. The proof of this theorem is contained in Theorem 4. 2 when the vector z e is replaced by the vector r e C.

142 In summary, then, we have shown that the best Chebyshev approximation in Uq will also represent a best Chebyshev approximation in a (2n+1)-dimensional reference subspace Un of. It should be mentioned that this result does not depend on the dimensionality of the reference subspace. It may be generalized to any m-dimensional reference subspace Um of Uq where c E { z = 1,2, C (Q)} and26 m > n + 1. We have selected m to be equal to (2n+l) because if, in each (2n+l)-dimensional reference subspace Uk of U, the error vector, e e U has all its (2n+1) components equal in absolute value to a parameter, I p, then the k-th vector relation of Eq. 4. 48 represents (2n+1) equations in (2n+1) unknowns, which are given by p and the 2n-parameters of, z). The problem of finding the best Chebyshev approximation in each (2n+l)-dimensional reference subspace of Uq will be considered in the next section. 4. 5 The Best Chebyshev Approximation in a (2n+l)-Dimensional Reference Subspace This section considers the problem of determining the best Chebyshev k-th reference solution vector, rk* e, which is de27 fined by Definition 4. 2, namely, to minimize the function Recall, from Section 2. 3. 1.2 (Chapter II) that for each prescribed z e 5 the reference subspace is (n+l)-dimensional. 27The term minimum is synonymous with the term "absolute minimum, " unless otherwise indicated.

143 (k) (k) 2n+1 IL (r)l_ cwith respect to r e 4, where r) Uk,subject to [A(k)(r)]T f(k) [A(k)(r)] (k) (4. 64) where f(k is a fully prescribed real vector in U n+, and [A(k)()] is a (2n+l) x (n+l) matrix defined by Eq. 4. 59 (or equivalently Eq. C.23). Our discussion will be primarily concerned with determining the vector r e 4? which yields the minimal value of \le(k)(r)\, subject to the constraint that all the (2n+l)-components of e(r) e Un+l are equal in absolute value to l (k) (r)llo We 28 shall denote this vector r by rk*, if there exists no r e, (k) which yields a smaller value of e(k)(r)lcc when only 2n-components of e(k) e U2n+l are equal in absolute value. If, however, there exists an r e e which yields a smaller value of Ie (k)(r)l when 2ncomponents of i (k)() e Uk1 are equal in absolute value, then we shall denote by rk* the vector r e. which yields the minimal value of (k) 11.. (r)l_ for such a case. The following points should be noted concerning the development presented in this section: Note that we use the same notation as that used in Definition 4. 2 since we assume (although we were not able to prove it) that such a vector r e ~R is the best Chebyshev k-th reference solution vector ins-.

144 (1) The vector r represents a vector in E, the (n+l)-dimensional Euclidian space, subject to the constraint that one of its components is equal to one. Specifically, we shall assume that the (n+l)-st n+l1 29 component of r e E is equal to one,, i. e., r = 1, so that for each prescribed vector (k) e U2n+l the (n+l)-equations given by Eq. 4. 64 are functions of the n-variables {r, rl,..., rn} If under this assumption one cannot obtain a finite vector r which satisfies Eq. 4. 64, then one must assume that some other component of r 30 is equal to one. n+ 1 (2) The components of the vector y(r) e E, defined by (r) = [A(k)( T ( (fk)_ (k)) (r) = [A (r)] ( ) are continuous functions of r c A, for each real vector e(k) e U k This condition follows from the fact that the nonzero elements of the matrix [A(k)(r)] are polynomials in terms of the components of r. At this point, let us note that Eq. 4. 64 represents a set of (n+l)simultaneous equations in (3n+l) unknowns, where the (3n+l) unknowns are given by the n-components of the vector r and the (2n+1) components of the vector () e U2n+ Hence, it is reasonable that if the 29-. 29This constraint guarantees that the polynomial equation given by Eq. 3.4, yields n roots. Clearly, this assumption represents the constraint that the vector r e En+l is restricted to the set llrll = 1.

145 values of 2n-unknowns are initially prescribed, then the value of the other (n+1)-unknowns can be determined from Eq. 4. 64. For example, for each prescribed r e 1 and n-components of the (k) 2n+l (k) vector (k) e Uk, the other (n+l)-components of ek) can always be determined from Eq. 4. 64. It should be noted, however, that in the case when 2n-components of (k) E Un+ are initially prescribed, a solution to Eq. 4. 64 may not exist. This is because in such a case Eq. 4. 64 may represent a system of (n+l)-simultaneous nonlinear equations in the n-components of r and the (2n+1)-st component of e(k) Now before presenting the method which we shall use to determine r * e A, let us consider the relations which we shall need. First, let us recall that for each r E W there exists a vector _,(k).2n+l 2n+1 e*(k) e U2n 1which satisfies Eq. 4.64 with a 2n+ -norm that is k- oC0 31 minimum. Hence, the problem of minimizing the function (k(r))l_ with respect to r e, subject to the constraint of Eq. 4. 64, may be replaced by the problem of minimizing the f(k) function \\* ()r)\\ with respect to r subject to the same constraint. Therefore, we shall seek the vector r* e A, such that 31 Recall that since Eq. 4. 64 corresponds to the k-th relation of Eq. 4. 47, then when r is initially prescribed in A, i. e., z is initially prescribed in, we have the Chebyshev approximation problem discussed in Section 2. 3. 1. 2 (Chapter II) when q= 2n+l and the matrix [E] is given by the matrix [Z'k)]. Note that the vector E*(k) denotes the vector ek) with ll_*(k)11 < 1 l(k)I for all e satisfying Eq. 4. 64, when r is initially prescribed.

146 the 2 +l-norm of the corresponding error vector, e*(k)(r), is given by _(k) min (k) Fle *(k)(k* )lIlx = m~ig, l(k)(_r) (4. 65) *k)(r~*oo = 00 r l *lloo 0 Since the vector e*(k)(r) is characterized by having at least (n+l)(k) component equal in absolute value to II (k)(r)ll, then the vector e*(k)(r*) must be characterized by having m-components equal in =k absolute valueto I e*(k)()11r, where n+1 < m < 2n+1. (k) To show that one may obtain a vector e*(k(r), with (2n+l)(k) components equal in absolute value to e (r)) _1, let us consider (k) Eq. 4. 64 when the vector EF (r) is given by (k) p ak) (4.66 e^ =por (4.66) (k) where now p is a real parameter and, the vector ) represents (k) an initially prescribed sign configuration, i.e., (k = I, i = 1,..,2n+1. Substituting Eq. 4.66 into Eq. 4.64, yields (k) 0 ( [A(k)()]T (f(k) p ) 0 (4.67) or alternately (k) (k) (k) (k) (X(r) f(k) =p(X)(r), (k)) j 1,..n+l (4.68) where ( (r) is a vector in Uk which represents the j-th column of the matrix [A(k)(r)]. Observe, that Eq. 4. 68 represents a system

147 of (n+1)-equations in (n+l)-unknowns, where the (n+l)-unknowns are given by the parameter p and the n-unknowns of the vector r. Since the parameter p is common to all the (n+1)-equations, then Eq. 4. 68 may be written as (X(k)(r) f(k)) (k)1(r) f(k)),,.,n -(4. 69) (X(k)(r) a(k)) (A(k) (k)) j = 1,..,n (4.69) (~_j _ _ (kn+l( and P (k) (k)) (4.70) ( Hence(k) the vector ( is firr)st determined(k)) if (X(k)(_r) a(k)) # 0. Hence, the vector r is first determined32 from Eq. 4. 69 and then knowing r the parameter p is determined from Eq. 4. 70. Since the nonzero components of the set of vectors (r) e Un: j = 1,...,n+l} are some polynomials in the variables {r, rl,.d te v r and the vectors and are initially prescribed real vectors, then Eq. 4. 69 can be represented as a system of n-polynomial equations n+l(r) D.(r) - N(r) D (r) = 0 j = 1 n (4 71) 1(1)1 n N+1-()= 0,,...,n (4.71) 32 ~ For the present discussion we shall assume that there exists a solution vector r to Eq. 4. 69 and that for this r the value of

148 where N.() (((r), f(k)) j =,...,n+ D.(r) = ( (r), ) j = 1,...n+ Let us now digress to present the following illustrative 33 example: Example 4. 3: Given the vectors ^+1 f2 2n f2n+ i. e., the components of a(k) are given by ok) = (-)i+1 i =1,...,2n+1. Furthermore, the (2n+1) x (n+l) matrix [A(k)(r)] is given by This example considers the case when the subspace Ukn+l contains consecutive (2n+l) components of the vectors in Uq.

149 r0 0... 0 ro 0 o^1 1 rn- 0 [A))] = I 1... r 0 1 r1 rn-l 0 0 0 0.. 1 Substituting into Eq. 4. 68, yields r. fi+ + f. =P (-1)+ r + (-1)n++ j,...,n i=0 1 i+J j+n fl i=O i=O Let n-1 N.(r) = r f. +f. jn+1 i=0j 1+] j+n and let D.() = (()r.)iirthe theabvei+eq in 1 y D.(r) = (-l~j [E (-1 ri + (-1)n+ ti=o L then the above equation yields

150 N.(r) = p Dj(r), j= or N.(r) p D ( ) j = 1,...,n+l Therefore, the vector r must satisfy the following system of equations N.(r) N. (r) 1- j+1 D.(r) D.( j = Since D.(r) = -D. (r), then we have N.(r) + Nj+(r) =, j =,.,n which yield the following system of n-equation n-l Z ri(fi+j +fi+j+) (fn+ + fn+j+l) 0 j =.,n i=O Solving this system for the n-unknowns {rO,..,rn } yields an r c. It suffices to say that if there exists no r c 1 which satisfies Eq. 4.71, then there exist no vector *(k)r) with (2n+l)-components equal in absolute value for the prescribed sign configuration vector a(k). If, on the other hand, there is more than one possible

151 solution vector r to Eq. 4. 71, then only the one which gives the smallest value of Ipl is of interest. Furthermore, since the solution of Eq. 4. 67 (or equivalently Eq. 4. 71) depends on the choice of the sign configuration vector a(k), then to determine the vector e*(k)(r) with (2n+1) components equal in absolute value to e*(k)(r) 1 and (k) with l e*(k)(r)lI minimum, one must solve Eq. 4.67 for all possible 34 sign configuration vectors, and then select the one which yields the minimal value of Ipl Definition 4. 3: Let the set of vectors {r. e Y } and the set of vectors E*(k)(rj 2n+1 4vectors ) e U } satisfy Eq 4.64 when *(k)(r.) =pj), where the set {(a(j) 2n+ } represents 6-j Ukn} represents -J 1 35 the set of 2n- 1 distinct sign configuration vectors. The vector rM e t denotes the vector in the set {r } which yields the minimal value of Il (rj)l1, and the vector e*() (rM) Un+l denotes the corresponding vector in the set {e(k()}. Let us now give the conditions under which we say that the vector rM e i is the vector r* e W defined by Definition 4. 2. -M ~k Conjecture 4. 1 Given anerror vector e*(k ) e 2n+ with (r M _ kl (2n+l) components equal in absolute value, according to Definition 4.3, and the corresponding vector rM e F which Note that there are 2 distinct sign configuration vectors, (k) 35Namely the i-th component of o(i) is given by +1 or -1.

152 satisfies Eq. 4. 64. The vector rM is the best Chebyshev k-th reference solution vector, r * if for every set of 2n- ~k components of E(k) e U,n+l denoted by {ic): v = I,...,2n}, and given by (k) 6 (rM), v = 1,...,2n (4.72) v v where 0 < < 1; the solution of Eq. 4. 64 yields either: (1) a complex r, i. e., an r or; or (2) a real vector r' e, for which e ) I, the 2n+ 1 (k) absolute value of the (2n+l)-st component of (k), satisfie s ^ (k )>lll*)(^)llo _ (k)43) 2n+E! (- 11c (4. 73) 2n+l Remark: Conjecture 4. 1 gives a test which guarantees that if there exists a vector rM E? (and it has in every example considered), then there exists no other *(k) r e F which yields a smaller value of IlL*(k ))l when only 2n-components of e*(k)) Uk2n are equal in absolute value. To prove whether such a test is sufficient for rM E - to be the vector rk* defined by Definition 4. 2, one must show that if Eq. 4.73 is satisfied, then there exists no r e

153 which yields a smaller value of e*(k)(r) when _ (Dl_, when less than 2n-components of e* (k) ) e U2n are equal in absolute value. Although we were not able to prove this, such a case was not encountered in the examples considered. Since the test, given by Conjecture 4. 1, involves solving Eq. 4. 64 when 2n-components of (k) e Un+1 are initially prescribed, we begin by separating the problem of finding the vector r c? from the problem of finding the unknown component of e(k) e Un+1 for such a case. Let us denote by the vector (V) the 2n-components of the vector e(k) e U2n+l which are initially prescribed. Clearly, (V) represents the projection of e(k) Ukn+ onto a 2n-dimensional subspace 2n of Ukn+ which will be denoted by U. There is no loss of generality in assuming that the vector e(V) E Un represents all but the v-th component of (k) e Un+l. Since there are (2n+1) different ways of selecting 2n-components from the (2n+l)-components of e (k) Uk, then we must consider the (2n+l) vectors in the set {e) e Uv: v = 1,...,2n+1}. Denoting by f(V) the projection of f(k) e U2n+l onto U2n then the relation between f(V) and e) in k _ 2n each U, v = 1,...,2n+1, is given by [V)(r f(V)= [A(^)r T () (4. 74) [A(V)(r)] T f_(V) [A (D)r] Tiv

154 where [A ()(r)] is a 2n x n matrix representing a submatrix of the (2n+1) x (n+1) matrix which is equivalent3 to [A (k))]. Furthermore, e(k) the v-th component of the vector e (k e Un (i. e., the unknown component of (k) when the other 2n-components are initially prescribed) is given by (k) 2n+1. (r)( (k) (k)(k) (k) ( ) V V i=1. (f rE (4. 75) (k) V, ( i i=1 (kjr) i/v /'Y where { k)(r): i = 1,..,2n+1} represents the elements of the j-th column of the (2n+1) x (n+l) matrix [A(k)(r)] for which (k).(r) when r is known. Observe that when e() = e*()(rM), then for all v = 1,...,2n+1, the vector rM must satisfy Eq. 4. 74, and Eq. 4. 75 must yield that ek) = e ((r Furthermore, observe that when Eq. 4. 74 has no solution, i. e., there exist no real r e, for some (v) U2n initially prescribed vector (^) U, then there exists no component e of the vector e e EU. At this point let us consider the following question: Suppose there exists an r e R such that the inequality of Eq. 4. 73 is not (k) 2n+l satisfied for some subset of 2n components of e(k) e Uk, i. k suppose that for some j e {v = 1,...,2n+1}, the absolute value The relation between the matrices [A( )(r)] and [A(k)(r)] is similar to the relation between the matrices [A(k)(r)] and [R(r)] discussed in Appendix C.

155 (k) (k) of e(k(r ), the i-thcomponentof E(k, which is determined from 37 Eq. 4.75 when r =r satisfies e (r) < Ie (rk ) lor (4.76) Clearly, such a case must occur when the best Chebyshev k-th referec lto rkE_ * 2n+1 ence solution, k* e A, yields an error vector e*(k)(r) e Uk which is characterized by having at most 2n-components equal in absolute value to 11 e*k(rk*)l i ()<(k) 2n+ Let us now consider the case when 2n-components of (k) e 2n+ are equal in absolute value to Ipl, where p is some real parameter in the interval (-c, cc). In other words, let the projection of the vecto ) r 2nk 2n 2n+' tor e U E n U2nonto the 2n-dimensional subspace U2 of U2+ where U e { v = 1,..,2n+1}, is given by () = p ( e 2n (4.77) where a() is a 2n-dimensional vector representing a prescribed sign configuration (i.e., a() = + 1 i =,..,2n) and where p is a real parameter in (-cc, oc ). Substituting Eq. 4. 77 into the i-th relation of Eq. 4. 74, yields [A ()(r)]T (f(l) - p ()) 0 (4.78) 37Note that rJ represents a vector in the set {rv}, where each r represents the solution of the v-th relation of Eq. 4. 74, which yields the smallest value of e (r ) I, given by Eq. 4. 75.

156 where [A()(r)] is a 2n x n matrix and f(') is the projection of the prescribed real vector f(k) e Uk onto U. k Observe that Eq. 4. 78 represents a set of n-simultaneous equations in (n+l)-unknowns, which are given by p and the n-unknown components of r, i. e., we have an extra degree of freedom when solving Eq. 4. 78 for p and r. However, since we seek the r e which yields the minimal value of Ile(k)(r), when e (k) e U2n+ (k) has 2n-components equal in absolute value to (k)(r), then we solve Eq. 4.78 for the vector r e 9 which yields the minimal value of I p. It should be noted that although for each value of p e (-cc, c), Eq. 4. 78 represents a set of n-simultaneous equations in the n-unknowns of r, there may be a case when for some p e (-oc,oc), there exists no real r (i.e., no r e?)which satisfies Eq. 4. 78, since in general Eq. 4.78 represents a set of 39 n-nonlinear equations. This is illustrated in the following example. Example 4.4: Given n = 2, 0 -1 i2 ~1i 1I -1 1 +1 3See Eq. 4. 74 for the definition of the matrix [A( )]. 39The example presented has been taken from Example 5. 3 of Chapter V.

157 and r O 0 [A( (r)]) r= -rl rl 0 1 Substituting the above value of f(), a(l), and [A (r)] into Eq. 4. 78, yields the following two equations: p rO + (2- p)(r- r12) + (1+p) r1 = (2- p) r0 + (1- p) r + (- p) = 0 Solving these equations for r0 in terms of p we have r (5p2 - lp + 2) ~ (1 + p) (5p- 1) (2p - 5p + 4 2p3 -4p+ 13p- 8 Note that r0 is complex if for some real p e (-o, o ) (5p - 1) (2p2 -5p + 4) < 0 Since the term (2p2 - 5p + 4) > 0 for all real p e (-oo,oo), and the term (5p- 1) < 0 for all real p < 0.2, then we note that there exists no r e? which satisfies the above equations for any p < 0.2.

158 Definition 4. 4: Let p0 denote the p e (-oo, cc) which yields the minimal value of I pl for which there exists an r e satisfying Eq. 4. 78. Let the vector r denote the vector in I which satisfies Eq. 4. 78 when p = pO and the vec(k) 2n~k tor E (k)(r U2n+1 denotes the error vector which satisfies Eq. 4. 64, when r = r0 and when the 2n-components of e(k are equal in absolute value to plp. Note that if the relation of Eq. 4. 76 is satisifed when r = r0 then the 2n+.1-norm of e*()(r) is given by *(k) lo= E (o = r Although one can obtain a test, similar to that given by Conjecture 4.1, which guarantees that there exists no r e 4E which yields a lower value of 11*(k()l1 than lli *()}(o) when only (2n-1)-components of i*()(r) e Ukn1 are equal in absolute value to IE *((r)ll. However, when (2n-1)-components of (k) e Un+k are initially prescribed to be equal in absolute value, then Eq. 4. 64 represents a 40 system of (n- 1) equations. in the n-components of r. Clearly, such a system contains more unknowns than equations, so that one has some freedom in selecting the solution vector r e i. Although the solution vector r should be selected to give the minimal value of.- D~ -....: This system of (n- 1)-equations is obtained from Eq. 4. 64 in a manner similar to that which obtained the system of n-equations given by Eq. 4. 74.

159 (k) 41 (k)(r)l_, it is not obvious how to select such a vector r. Hence, we shall say that if Eq. 4. 76 is satisfied when r = rO, then the vector r0 e e represents the best Chebyshev k-th reference solution vector r*. -k This is summarized in the following conjecture: Conjecture 4. 2 Given an error vector *(k)(r0) e Ukn according to Definition 4. 4, i. e., *(k)(r,.. ei -0 ) = P0u^i i j / = 1,..., 2n+. E (r U 1). 2n+l 0) < e*(k)(r0) < I po where a. = ~1, and the corresponding vector r e o? which satisfy Eq. 4. 64. The vector r9 is the best Chebyshev k-th reference solution vector r*, if ll*(k)(rO)llo < E* (k)(r)lc (4. 79 where e*(k)(r) is defined by Definition 4. 3. In Example 5. 3 of Chapter V, we have selected the solution vector r which minimizes the absolute value of the unknown compo- nentof el)(r) e U2n. However, it should be noted that we were unable to show that such a method of selection yields the minimal value of IIl(k)(r)1 Hc

160 Now that the preliminary ideas have been discussed let us outline 42 the procedure we shall use to determine the vector rk* e: First, we determine the best Chebyshev error vector e*(k) e Un for some initial estimate of r e. Then we systematically decrease the value of Ile *k)1 and determine the corresponding value of r e R. Eventually it may happen (and it has in every example) that all the (2n+l)-components of e*(k) e Ukn+ are equal in absolute value to lI *() for some r c s. At this point we apply the test 43 of Conjecture 4. 1. If the test fails we then decrease the value of I e*(ik) until either (1) the conditions of Conjecture 4. 1 are satisfied, or (2) the vector r e 6, which yields the minimum value of llI*(k)r)l when 2n-components of * (r) e U are equal in absolute value to Ille(k)(r)l, is obtained. When either one of these cases occurs we shall say that the resulting vector r e? is the vector r * e *. -k Before concluding the discussion of the best Chebyshev approximation in Un+l, let us make some observations concerning the sign k configuration of the components of ** (k) which are equal in absolute 42 The detailed consideration of this procedure is given in Section 5. 3 of Chapter V. Note that at this point.the prescribed sign configuration of the vector e*(k) may not represent the sign configuration which yields the minimal value of ll *(k)(r)lloc when (2n+l)-components of e* k)(r) e Ukn+l are equal in absolute value to Ill*(k)(r)l\\

161 value to I e**(k)1. Recall from Corollary 2. 1 that for each prescribed vector z e 3 (or equivalently r e. ), the best Chebyshev 2n+l1 approximation in U, also represents the best Chebyshev approxi2n+1 mation in some (n+l)-dimensional subspace of Uk. In other words, if for a prescribed r e i, *(k) represents the best 2n+1. (0) Chebyshev error vector in Uk, and if e* represents the best Chebyshev error vector in some (n+1)-dimensional reference subn+1 2n+1 44 2n+1 space U of U where from Eq. 2.65, we have -(k) max (0) ll* lc = 1l(C;- (4.80) 11^ 11.0 2+1,111*n11. i4 *0) ~< ~ n+1 Furthermore, recall from Theorem 2. 3, that in each (n+l)-dimensional subspace Un+, the vector * (0) is given by (0) (0) where (f(0) X( ) P= ( (4.81) and (0) (0) ai = sgn, i=...,n+l (4.82) Note that there are (2n+l) distinct (n+1)-dimensional subspaces in the 2n+1-dimensional space U2n+l.

162 and where f(8) represents the projection of the vectorp f(k) E Uk n+l (0) n+1 onto U, and X is a real vector in U0 which is determined from Eq. 2.41. Since lle*() = ipl then Eq. 4. 80maybe written as II*^llma = {2n+l{ {Ipl } (4.83) - 1 < n+l By examining the relation between the (2n+1) x n matrix [Z(k(z)] and the (2n+l) x (n+l) matrix [A(k)(r)], it is noted that, for each r e., the nonzero elements of the (n+l)-column vectors of [A(k)] 45 (f) represent a particular set of (n+l)-vectors in the set {X(): 6 = 1, 2n+l1. n. (n+)} Furthermore, for each n+l-reference subspace in the set n+1 2n+1 ( { U: I,..., ( n+l)} the corresponding vector _( can be obtained by taking the linear combination of the column vectors of the (k) matrix [A (k)] so that the n-components of the resulting vector which n+1 46 do not lie in Ul are equal to zero. Observe that since the matrix [A(k)(r)] depends on the vector r c I, then the set of vectors { ()'} will also depend on r. To emphasize this dependence we shall denote X(0) by X(0(r). Substituting X(0)(r) for (0) in Eq. 4. 81 yields 4Note that the nonzero elements of each column vector of [A(k)] represent the vector x(0) when the corresponding (n+l)-dimensional reference space contains the (n+l) consecutive components of the vectors in Ukn+ 6This will be clarified in Chapter V, where numerical examples are presented. - - -

163 (f(), X( )(r)) P ") (- ~ )~1 (4.84) This expression relates the minimal value of lE ()(r)\I = Ip(r) I in n+ 1 2n+l1 each (n+l) dimensional subspace UO, 0 = 1,.., (2n),to the t/ n+ i vector r e i. Hence, when r is not initially prescribed, then the (k) value of { (k)], given by Eq. 4. 83, becomes 11e*(k) = man {1P0(r)l} (4.85) ll^^lloo- 0 "2n+l{) n+ 1 where p0(r) is given by Eq. 4. 84. Hence, the problem of minimizing the function e*((r)l 0, stated in Eq. 4.65, may be written as E. (k) _r min max *)r11- r E~!e<_ (2n+) ()ko = re ~ {1p2n+l {Ip0(r)} (4.86) This representation of the minimization procedure yields the (r) following geometrical interpretation: If we denote by X ()r) the vector x0)(r)/ll A )(r)111, then it is noted from Eq. 4. 84 that each function pc(r) represents the orthogonal projection of the vector ) n+l ( f(I) e Un onto the vector X (r). Since, by definition, the vector (8' n+l X ()(r) is orthogonal to the approximating subspace, in U+ (i.e., the column space of the (n+l) x n matrix [Z() (z)] ), then by varying the vector r, one essentially rotates the vector (0)(r) (or equivalently the approximating subspace). Clearly in each U~ we may 0 -..

164 rotate X ()(r) until P8(r) = 0; however this may result in P(r) f 0 in some other (n+l)-dimensional subspace. Thus, -the minimizing procedure seeks the r e 4 for which the vectors in the set {X6(r)~): 1,~~~12n+1 {(0 )(r): = 1. (2n+l)} will yield a corresponding set of values { 1p(r) I} with the largest value being minimum. It has been found, in the examples considered, that when r = r*, then this minimum will occur when at least (n+l) functions from the set { Ip(r) I} are equal. Recall that since each IP(r) I represents the minimal value of e*(0)(r) llt in Un+, then when (n+l) functions from the set { I(r) I } are equal for some given r e I, they represent the minimal value of o -norm of the error vector in some higher dimensional subspace defined by n+l U Un+ (4.87) j=l j n+l where {US } represents the set of (n+l)-dimensional subspaces in j which the functions in the set { Ip0j(r) I } are equal. Furthremore, it has been found that the number of components of the error vector e*(k)(r*) e2 U which are equal in absolute value to 1{e*(k)(r}ll -k k0 is directly related to the dimension of the subspace defined by Eq. 4. 87. For example, if only 2n-components of (k)(r*) are equal in absolute value to Il_*(k) (k*)\, then Eq. 4. 87 will define a 2n-dimensional subspace of Ukn.

165 In summary, then, it has been found that the projection of the (k) components of *(k)(rk*) which are equal in absolute value to II*(r * for the given r * e, represent the best Chebyshev error vectors in at least (n+l) subspaces {U } of Un+ where n+eco theI sie in each of these subspaces Uj, the sign configuration of the vector Ij)' J 47 e*(0J)(rk*) is given by Eq. 4.82, that is,7 sgn ei (r = [snp(k)] sgn rk)] i =.,n+1 1(8j)k 0-:-k snP1 (rk*) j = 1,...,n+1 (4.88) It should be mentioned that in the case when (2n+l) components of **k e Ukn are equal in absolute value to.. **(k) l1, it has been 48 found that the signs of these components alternate, namely, ~ (k) **(k) sgn e+1 = -sgn e i = 1,...,2n+l Note that such a sign alternation property is not encountered in the case when the vector r e is initially prescribed. 47 4Note that this result is based on the examples considered in Section 5. 4, (Chapter V) and it has not been proved whether the sign configuration given by Eq. 4.88 is sufficient for le*)(rk)l to be minimum. 48 See Examples 5. 1 and 5.2 of Section 5. 4, Chapter V. 49 49See the sign configuration given by Eq. 4. $2, or equivalently by Theorem 2. 3 of Section 2.3.1.2, Chapter II.

166 4. 6 On the Uniqueness of the Best Chebyshev Approximation Although we are primarily interested in the existence and characterization of the best Chebyshev approximation, let us make some observations concerning the uniqueness problem. Since the uniqueness of the best Chebyshev approximating vector, f** e Uq, defined by Eq. 4. 1, depends on the uniqueness of the vector pair (**,z*) Fe z x,, we state the following theorem: Theorem 4.4: The best Chebyshev real approximating vector f** e Uq, defined by Eq. 4.1, is unique if the best Chebyshev parameter vector z* c 3 is unique50 and its n-components satisfy: (1) Zk* 0 for allk =,...,n; and (2) zk* / j for all k = 1,...,n, where C is a real number, i. e., no component is purely imaginary. Proof: Recall from Section 2. 3. 1.2 that if every (n x n) submatrix of the prescribed q x n matrix [E] is nonsingular, then the best Chebyshev approximating vector f* Uq is unique. Since the conditions (1) and (2) of the theorem guarantee Assumption 2. 1 (i. e., Since the vector z e ~ represents an arbitrary ordering of the set of n-parameters {zk},then the vector z e 3 is unique if the elements {zk} are unique.

167 that every (n x n) submatrix of the (q x n) matrix [Z(z*)], defined by Eq. 1. 18 (Eq. 1. 18a), is nonsingular), then the theorem is proved. In terms of "Prony's Extended Method" Theorem 4. 4 becomes: Theorem 4. 5: The best Chebyshev real approximating vector f** e Uq, defined by Eq. 4. 1, is unique if the best Chebyshev solution vector r* e i, defined by Definition 4. 1, is unique and it yields a unique error vector e *(r*) e Uq. Proof: All of this theorem is contained in Theorems 4. 4 and 3. 1. Recall that Theorem 3. 1 gives an alternate way of representing the best Chebyshev approximation in terms of the vector r* e i and c*(r*) eU. By applying Theorem 4. 3 to Theorem 4. 5 we have the following theorem: Theorem 4. 6: The best Chebyshev real approximating vector f** e Uq, defined by Eq. 4.1, is unique if the best Chebyshev approximation in the (2n+l)-dimensional reference subspace U2n+l is unique, i. e., the best Chebyshev reference c-th solution vector r * e 9 is unique and it yields a unique error vecCtor tor E*(C)(r*)e U2n+l. ~ ^c' c

168 It should be noted that Theorem 4. 6 gives a stronger condition on the uniqueness of the best Chebyshev approximation. In other words, it is possible to obtain more than one best Chebyshev c-th reference solution vector r * E 9 which yields a unique vector *(C)(r *) and still obtain a unique Chebyshev approximation in U4 if the solution vectors r * are distinct. 51 -C 52 The following example illustrates the case when the best 2n+1 Chebyshev approximation in U2 yields two distinct vectors r *(1) and r (2) in? which define the same error vector -k k e *(k)(r *) e U2n+l Example 4.5: Consider the case when n = 1 (i.e., the vector z = z), and the (2n+l)-dimensional reference subspace Uk (i. e., Uk3) in which the approximating vector is given by f(k) 2 f (, Z) =ZI " ~~~~~~~~~~51.~~~~~~~ 51This results from Theorem C. 1 where we have shown that for each prescribed r E 4W and E(k U n+l, there exists a unique vector c e Uq. 52The example considered here is based on the numerical example given by Example 5. 1 of Chapter V.

169 Let the prescribed vector f(k) e Uk be given by 1. 0000 f (k) 0. 2500 0. 0625 then Eq. 4. 48 yields the relation 1. 0000 I e( 1(k) 0.2500 = z2 + 2() 0. 0625 z e(k) and Eq. 4. 64 yields the relation (k) 2 2 0 1. 0000 2 2 (k) 0.2500 = 1 1' 0 r 4 -r 4 0 r 4 -r 4 r -r1 r0 -rl 0.0625 -r (k (4.89) 3 since the 3 x 2 matrix [A (k)(r)], obtained from Definition C. 3, is given by r 0 0 0

170 where r 0 r Let us assume that the best Chebyshev error vector e*(k)(rk*) e U3, is given by E*(k)(r_~) = p.(k) *O(rk*)=pa where +1 v(k) =-1 (k). Replacing the vector e_ in Eq. 4. 89 by the vector I *(k)(r) given above, yields the following system of simultaneous equations: P(rO +rl) = r 0.25 r -P(ro + r1) 0.25 r 0 0625 r (4. 90) Letting r = 1 and solving Eq. 4. 90 for r1 and p yields r0 = ~ 0. 546 and p = 0. 037. Hence, we have two best Chebyshev k-th reference solution vectors, given by

171 r = 0. 546 -0. 546 =* -, and (2 = 1. 000 1. 000 which yields the following unique error vector in Uk3 +0.037 1*(k)(r*) = -0. 037 +0. 037 In conclusion, it should be mentioned that it has been found that when the best Chebyshev approximation did not yield an error vector C ** e Uqwhich has (2n+1) components equal in absolute value to II ** II, then the best Chebyshev approximation was not unique. Specifically, Example 5. 3, given in Section 5. 4 (Chapter V) depicts the case when the best Chebyshev approximation yields an error vector e** e Uq which has only 2n-components equal in absolute value to {11 ** tl C In this case, one of the components of the vector z* e _ were found to be zero so that the corresponding component of the vector I** e Sc may be selected arbitrarily. Hence, it may follow that the error vector e**, which has (2n+l)-components equal in absolute value to \11 ** 1l:,. characterizes a unique best Chebyshev approximation. Since we are primarily interested in the existence problem, we shall not examine this facet of the problem further.

172 4.7 Summary The existence of the best Chebyshev approximation to the problem defined by Eqs. 4. 1 and 4. 2 has been shown in Section 4.3. Furthermore, we have obtained the bound within which the value of 1LiE** oc must lie. Sections 4. 4 and 4. 5 discussed the theory behind the method of finding the best Chebyshev approximation, i. e., it has been shown that the best Chebyshev approximation can be obtained by systematically decreasing the value of eI*(r)lc. This result will be used in Chapter V to obtain an algorithm for finding the best Chebyshev approximation.

CHAPTER V COMPUTATIONAL METHOD AND EXAMPLES 5. 1 Introduction The purpose of this chapter is to give a computational method which will yield the best Chebyshev approximating real vector f** = [Z(z*)] ** to the prescribed real vector f Uq. Our approach to the solution of this problem is based on the theory developed in Chapter IV, namely a method of descent. We shall begin this chapter by briefly discussing the methods of descent. Then, based on the theory of Chapter IV, an algorithm will be presented in Section 5. 3. Numerical examples illustrating this algorithm will be given in Section 5. 4. 5. 2 The Method of Descent As is well known, the method of descent is a systematic method used in solving minimization problems. It specifies a way of reaching the minimum of a multivariable function, by determining the downward direction of the function and the distance one must travel along this direction. One of its drawbacks is that it does not always distinguish between a local minimum and the overall minimum. This problem, however, is not encountered when minimizing a function which is convex, the case that is usually found in the linear L (p )-approximation theory, where 1 < p < cc. A discussion of the method of descent and its 173

174 application to the theory of linear approximations is in Rice's book (Ref. 19, pp. 158-186). For example, he illustrates the use of the method of descent in solving Chebyshev approximation problems of the type discussed in Section 2. 3. 1. 2 (Chapter II), namely, the case when the vector z is initially prescribed. Let us now consider the method of descent that we shall use in solving the Chebyshev approximation problem given by Eqs. 4. 1 and 4. 2. That is, given a real vector f e Uq minimize the function Ie(3, z)II Ilf - [Z(z)] lI3 with respect to the vector pair (i, z) e x, such that (_**, z*) yields e ** IIE**, z*)l = min Ille(, z)U (R, Z)E z x Recall, from Section 4. 3, that since IIe** II = min 1e*(z) 11 - 00 O where e*(z) e (1*, z) e Uq, then we want to find the minimum of the function lle*(z) ll for all z E. In seeking the minimum of the function llE*(z)11 we shall use "Prony's Extended Method, " namely we shall minimize the function lIc*(r)II with respect to re, where the vector *(r) e Uq must satisfy [R(r)]Tf = [R(r)]Te*(r) (5.1)

175 and where the qx(q-n) matrix [R(r)] is defined by Eq. 3. 2. Hence, the method of descent will seek the vector r* e I, such that lle**ll ^ ll (r*) ll min llm *(r)11 (5.2) l- 0 _**1100 =-"' ~ oc rE/ Although, in general, the function Ile*(r) 11 is not convex with respect to r e I, one can obtain a method of descent which yields the overall minimum of lle*(r)ll by using the fact that the vector r* e will yield the best Chebyshev approximation in some (2n+l)-dimensional 2n+ 1 o 2n+1 U2n+ reference subspace, Uc, of Uq where Uc e Uw:= 1,.. Ic ( 1) In other words, the selection of the appropriate (2n+l)-dimensional reference subspace of Uq will govern the direction of descent and the distance one must travel along this direction. Let us now outline the method of descent we shall use to determine the point r* defined by Eq. 5. 2. We begin by choosing an initial estimate of r e e, for example, the vector r'. Although the vector r' may be selected arbitrarily, recall from Lemma 4. 2 that if r' = r, where r is defined by Eq. 4. 39, then one can obtain the bounds within which the value of le** II_ must lie. Knowing the vector r' e I, the corresponding best Chebyshev error vector e*(r') e Uq can be found. 2 Since the vector e*(r') Uq has at least (n+l)-components equal in absolute See Eq. 4. 40, Chapter IV. Recall that for each prescribed r' ed, there is a corresponding z'?e, and so the qxn matrix [Z(z')] is fully prescribed. Thus, we have the Chebyshev approximation problem discussed in Section 2. 3. 1. 2, Chapter II.

176 value to lle*(r') II, then let us denote these components by the set3 { (r'):v= 1,. m; m> n+1. If m < 2n+1, then we decrease, in small steps, the absolute values of the m-components in the set e* E v and calculate a new value of r e IW, and the other (q-m)-components of the vector e e U. When the absolute value of one of the (q-m) components of e is greater than or equal to that of the component in the set ei: v = 1,..., m we add it to that set, i. e., we increase the value of m. Eventually, we obtain the case when m = 2n+1, i. e., we have an (2n+l)-dimensional reference subspace of Uq which satisfies Eq. 4.63 of Theorem 4. 3. To see if it satisfies Eq. 4. 62 of Theorem 4. 3, we apply the test given by Conjecture 4. 1. If the conditions of Conjecture 4. 1 are satisfied, then we have obtained the minimal value of lle*(r)ll and the corresponding r is the best Chebyshev solution vector r* e e. If, on the other hand, the conditions of Conjecture 4. 1 are not satisfied, then we decrease the absolute values of the set of 2n-components for which Eq. 4. 73 has failed. Eventually Conjecture 4. 1 (or Conjecture 4. 2) will be satisfied in some other (2n+l)-reference subspace. 5. 3 The Computational Procedure In this section we shall give an iterative procedure which yields the best Chebyshev approximation to the problem considered in Chapter IV. Note that this set can be used to define an m-dimensional reference subspace of Uq (see Appendix C).

177 Let us begin by listing the equations, in the form we shall need, as follows: (1) f = [Z(z)]_3~ + e_(B, z) in Uq (5.3) where f is a prescribed real vector in Uq; [Z(z)] is the (qxn) matrix, defined by Eq. 1. 18 (or Eq. 1. 18b); (3, z) is the parameter vector pair in ~ x; and e(1, z) is an unknown real error vector in Uq (2) f [Z] +e(C) in U (5.4) where [Z] is the (qxn) matrix [Z(z)] when the vector z e is initially prescribed. Note that Eq. 5. 4 represents the form of Eq. 5. 3 when z e is initially prescribed. (3) [R(r)] fR(r)R(r)]Te(r) in E-n (5.5) where (r) e(0, z) in E q; [R(r)] is the qx(q-n)-matrix defined by Eq. 3. 2; and where r is a vector in E n+given by r 0. r= r1 (5.6) r n J

178 Since we are interested only in the value of r within a constant, we shall assume that the component4 r = 1. An alternate representation of n Eq. 5. 5 is given by n ri(fi+- e+j) = 0, j = 1,..., q-n (5.5a) i=0 " (4) 6(r) = [F] r in Eqn (5.7) where [F] is the (q-n)x(n+l) matrix defined by Eq. 4.34; and 6(r) is an unknown real vector in Eqn. When seeking the minimum of II 6(r) with respect to r e E when IlrIl1 = 1, we shall use the relation min Il6(r )11il min 116(r)ll min () (5.8) llrII=1 0<j<n llrj II where r() denotes the vector r e En+ with the j-th component equal to one, i.e., r(j = 1 (5) When m-components of the vector E e Uq are equal in absolute value, where n+l < m < 2n+1, then the m-dimensional k-th reference subspace which contains these m-components is denoted by Ukm. Hence, according to Definition 2. 1 (Chapter II) the projection of e Uq onto Uik is If the approximation procedure indicates that the other n-components of r tend to become large (i. e., tend to infinity), then we shall select some other component from the set {:r,, r,..., r., } to be equal to one.

179 (k) denoted by e (k) and given by (k) _ (k) (5 9) vvhr- ~ p or (5. 9) where cr(k) is the vector in Uk representing the prescribed sign configuration of (k) i.e., (k) = 1, i=,..., m. Denoting by (k), q Mv (k) (k) the projection f e Uq onto Uk, then the relation between f and e k in Uk, is given by [1(k)]Tf) = [(k) (5.10) [A (r)]f [A (r)] (5.10) where [A (k)(r)] is an mx(m-n) matrix defined by Definition C. 3 (Appendix C). Its column space defines the orthogonal complement subspace of the approximating subspace in Um, and it is obtained from the mak' trix [R(r)] by appropriate column operations5 and partitions. Remark: When the vector E (k) Uk is given by Eq. 5.9, then Eq. 5. 10 will assume one of the following three forms: (1) If m = 2n+1, then Eq. 5. 10 represents a system of (n+l)-simultaneous equations. This system will be solved for p and the n-unknowns of the vector r (see Eqs. 4. 69 and 70). (2) If m = 2n, then Eq. 5. 10 represents a system 5By appropriate column operations we mean the operations on the columns of the matrix [R(r)] which yield a set of (m-n) independent vectors with (q-n)-components, not lying in UkM, that are equal to zero.

180 of n-simultaneous equations. This system will be solved for the n-unknown components of the vector r, by initially prescribing the value of:p (see Eq. 4. 78). (3) If m < 2n, then Eq.; 5. 10 represents a system (m- n)- simultaneous equations. This system will be solved for (m-n) components or r, by initially prescribing the value of p and the (2n+l-m) components of r. n n (6) P(z) = II (z-k) = r.z = 0 (5.11) k=1 i=0 where the ordered sets {r,..., rn and {z..., z represent the vectors r e r? and z e, respectively. (7) Ac - i(r(2) - e(r ), i= 1,..., q (5. 12) where e.(r )) and e.(r ) represent the i-th component of the vector e(r) e Uq evaluated at r = r(2) and r r(1) espetively. 5.3. 1 The Algorithm. (1) Given the value of the real vector f e Uq, and the values of q and n. (2) Using Eq. 5. 7, determine the vector r in the set lrII1 = 1 for which 116(r) IIc is minimum. Denote this vector by r' and the 6Note that since r' represents the initial estimate of r, it may be picked arbitrarily.

181 corresponding vector _(r) by 6(r'). If ll 6(r')ll = 0, then *(r') = 0 and r' = r* go to Step(31). Otherwise, Step (3) is next. (3) Determine the best Chebyshev real error vector E*(r') E Uq This is done by first finding the vector z' e 5 from Eq. 5. 11, when r = r' and forming the vector relation of Eq 5. 4. By using the procedure given in Appendix A, obtain the best Chebyshev approximation to the problem given by Eq 5. 4. The resulting error vector is the best Chebyshev error vector E*(r') Uq. II (r') 11 (4) Is [ril = lle *(r')II? If it is, then r' r* and proceed to Step (31). If it is not, go to Step (5). (5) Form the set {e* (r'): v = 1,..., m which contains the mcomponents of e*(r') Uq with absolute values that are equal to Ile*(r')11 Note that m > n+1. (6) Is m > 2n+l? If it is, go to Step (19). If it is not, proceed to Step (7). (7) Form an m-dimensional reference subspace, Um, which contains all the components of *(r') given by the set {E* (r'): v=,.. m found in Step (5). (8) Knowing Utm, determine the vectors f) and E (1)(r) in, which represent the respective projections of f and E (r) e U onto U. Furthermore, determine the mx(m-n) matrix [A ()(r)], which relates f(l) and e (1)(r) in U

182 (9) Form a new vector r, denoted by r" and defined by r0 r r" = m-n-1 mr-n r n where the last (2n+l-m)-components of r" take on the values of the last (2n+1-m)-components of r' found in the previous iteration step, and where the other (m-n)-components of r" are parameters. (1) (1) (1) (10) Let e((r") =0.9 [ ((r')], where e((r') denotes the vector l)(r) found in the previous iteration step. Then, using (1) and [A()(r)] ir r" found in Step (8), determine the value of r" (i.e., evaluate the set of parameters {r0, r1,..., rm-n }) from the system of simultaneous equations given by Eq. 5. 10. If there is a real solution, go to Step (12). If there is no real solution, go to Step (11). (11) Is m = 2n? If it is, go to Step (31). If it is not, repeat the procedure from Step (10) where now the vector r"' contains an extra parameters, i. e., it contains (m-n+l) parameters. For example, the fixed component rjn_n of the vector r", defined in Step 9, is replaced by a parameter denoted by rnan.

183 (12) Knowing the value of r" and the components of e ((r") which are in U1m, determine the vector e(r") in Uq using Eq. 5. 5 when r = r and f takes its prescribed value in U. (13) Is lle(r")llo = le(1)(r")ll? If it is, go to Step (14). If it is not, go to Step (15). (14) From the set {i(r"): i = 1,2,..., q} form the subset ei (r"): v = 1, 2,..., m which contains all the i's satisfying ei(r) I = 11e(r") cc. If this set is identical to that used in the previous iterative step, then add to it the component e (r") satisfying lA. I = max lAe. I j] 1 <_i<q 1 where Ac is given by Eq. 5. 12. Step (16) is next. (15) From the set e (r"): i= 1, 2,..., q form the subset ei (r): v = 1, 2,..., m} which contains all the i's satisfying I i(r") > v II (r'")11. Step (16) is next. (16) Is m = 2n+1 (where the value of m was found in Step (14) or (15)? If it is, go to Step (19). If not, go to Step (17). (17) Is m > 2n+l? f:itis, repeat from Step (10), by using the vector (r() = c (r'), where 0.9 < c < 1.0. If it is not, go to Step (18). (18) Form an m-dimensional reference subspace, U2, which contains all the components of e (r") e Uq given by the set {e (r"): = 1,..., m found in Step (14) or in Step (15). Step (8) is next, where the reference subspace Un of Step (8) is replaced by the reference subspace Um.

184 2n+ 1 (19) Form a (2n+l)-dimensional reference subspace, U2, so that it contains all the components of e (r") e U given by the set i.:v= 1, 2,..., 2n+1} found in Step (14) or in Step (15). V 2n+1 (2) (2) (20) Knowing U2, determine the vectors f(, e(r") in 2n+1 (2) 2n+ U2 and the (2n+1)x(n+1) matrix [A()(r)] which relate them in U2n+ Furthermore, determine the vector a(2) 2n+1 from the relation.(2) sgne(2)(r), i= 1,..., 2n+l. (21) Let (2)(r) = p(2) Then using f(2) and [A 2(r)] found in Step (20) solve Eq. 5. 10 for r and p. If there are more than one solution vectors r for the same values of p determine them all. 8 (2) (22) Knowing r and E (r), determine the vectors e(r) in U9 using Eq. 5. 5. Denote byr, the vector r which yields a vector e(r) in Step (22) with lle(r) II = Ip1, and byp, the corresponding value ofp found in Step (21). Proceed to Step (23) to test if r is the optimum r*. (23) Solve the equation used in Step (21) (i. e., Eq. 5. 10) for the set {r(':=1,,..., 2n+l and {p( ) when ) 0.99p(2) for i v (2) - p() for i = v, where i = 1,2,..., 2n+l, and where the value of p and ) were found in Step (21). In the next chapter when considering the application of this computational method to the time domain network synthesis problem, we shall restrict r to give a physically realizable network.

185 (24) Is Ip() > Ipi, for all v = 1,2,..., 2n+? If it is, then r = r* and go to Step (31). If it is not, then rr*, and go to Step (25). (25) Let ji denote the set of indices v for which Ip() I< lp I. Determine the vectors { (r() in Uq from Eq. 5. 5, by using r = r( and the values of (2)(r)) determined in Step (23). (26) Out of the set of vectors (r()) is there a vector (r)) such that li (ri)ll > Ipl? If there is, repeat from Step (15) using the vector E(r^()). If not, go to Step (27). (27) Out of the set { (r())}, pick any vector and denote it by E(r') and repeat from Step (5). (28) Determine the vector a(1) from the relation i(l) = sgn ei. (r), i = 1,...,2n, where E(1)') denotes the vector (l)(r) found in the previous step. (29) Let ()(r) = p a) whereo(1) is given in Step (28). Then using f1) and [A (r)] found in the previous step, solve Eq. 5. 10 for r and p so that the value of Ipl is minimum. (30) Knowing r and (1)(r), determine the vector e(r) in Uq by using Eq. 5. 5. If l (r) lo > 1i e(r) r = Ip, then repeat from Step (15). If 11 (r)11 = E()(r)II = Ipl then the vector r found in Step (29) is the optimum r*. Step (31) is next. (31) Knowing r* in E+ determine z* from Eq. 5. 11.

186 (32) Knowing z*, determine9 ** from any set of n equations defined by Eq. 5.3, where (C, z) = e(r*) found in Step (22), or Step (30). (33) From (**, z*), determine the vector f** [Z(z*)] ** 5. 3. 2 Comments on the Algorithm. The algorithm presented in Section 5. 3. 1 is based on the theoretical results obtained in Chapter IV. To program this algorithm, for digital computer use, is no simple task. The difficulty lies in attempting to program the steps containing the for(k) mation of the mx (m-n) matrix [A(r)], i. e., the steps which construct the system of simultaneous equations. To perform these steps the computer program must be able to generate its own program, i. e., the system of simultaneous equations in variable form. On the other hand, to specify initially all the possible forms of the elements of the matrix (k) [A(k)(r)] is impractical when n > 2 and the value of (q-n) is large. A more suitable procedure for use with a digital computer would be the one which minimizes the function le *(r) 11 with respect to n+ 1 o r c E n, along the set II r 1 = 1; or equivalently, the function IE *(z) ll with respect to z e 5. However, to develop such a procedure one must first obtain a method of varying r (or z) so that the value of Ile*(r)lII or Ile *(z) 11 is systematically decreased. An alternate procedure requires Note that the vector "* is not unique if some of the components of z* are identical, and/or equal to zero. Hence, if some of the components of b** can be made to equal zero without affecting the value of ie ** I o, then we may neglect the corresponding components of z**, i. e., lower the value of n.

187 the development of an exchange method [of the type developed by Stiefel (Ref. 22)] which will systematically exchange the (2n+1)-dimensional reference subspaces until the reference subspace U of Theorem 4. 3 (or Theorem 4. 2) is attained. 5.4 Numerical Examples The following examples illustrate the algorithm presented in Section 5. 3. 1. The first two examples chosen (i. e., Examples 5. 1 and 5. 2), are identical to those considered by Ruston (Ref. 20). It will be seen that a considerable improvement over his result is achieved using the above procedure. Furthermore, these two examples will illustrate the effect of increasing the dimensions of the parameter space on the resulting Chebyshev approximating error. The first example considers the simple case when n = 1, and it will be worked out in extensive detail. The second example, the case when n=2, will be presented using somewhat less detail. Example 5. 3 illustrates the case when the best Chebyshev approximation is characterized by an error vector E ** Uq which has only 2n- components equal in absolute value to IIe** II. Examples 5. 4 and 5. 5 illustrate the case when the components are not distinct. Example 5. 6 illustrates the case when the minimum value of lI *(r) I is attained at the vector r, with the component rn =0. Now before turning to the examples, it should be mentioned that a digital computer was used to solve these examples, however, it was used simply as a desk calculator because of the difficulty encountered

188 in programming the algorithm of Section 5. 3. 1. In the following examples, we shall denote by "( )" the computational steps, and by "step-" when referring to the steps of the algorithm of Section 5. 3. 1. Example 5. 1 Consider the problem of approximating the real vector 1. 0000 1 0.4450 z 0.2500 z 0. 1600 z3 f = 0.1110 by the real vector f(f, z) = z4 0.0817 z 0. 0625 z6 0.0494 z7 0. 0400 z so that lf - f (j** z*)II is minimum for all (, z) in Zx a. The computation procedure is as follows: (1) Step 1: Substitute the prescribed vectors into Eq. 5. 3. This yields 1.0000 1 61 0.4450 z 0.2500 z2 0. 1600 z 4 0.1110 = z + e in U9 (5.13) 0.0817 z E 0.0625 z E 0. 0494 z 0. 0400 z E Note that q=9 and n= 1.

189 (2) Step 2: Eq. 5.7 becomes 1.0000 0.4450 rj 0.4450 0.2500 0.2500 0.1600 r 0. 1600 0. 1110 ) - 0.1110 0.0817 A 0.0817 0.0625 0.0625 0.0494 0.0494 0.0400 The vector r, and the corresponding vector 6(r ) for which 116(r) 11 is minimum for all r in the set Ir II = 1 are -0.039 0.035 (1) 0.48 (1) 0.03 L 000J 0.0230 0.0192 0.0162 Note that when n= 1, Ruston's method will always yield the vector r for which II6(r)11l = minimum. (3) Step 3: From Eq. 5. 11 we get z() = 0.484. Substituting this value of z into Eq. 5. 13 and determining the best Chebyshev approximation, by using Stiefel's method (see Appendix A), yields

190 -0. 0331 -0. 05454 0. 0085 (1) *(z()) 0. 0432 E*(r ) = ( 0. 05454 0. 0544 0. 0493 0.4043 0. 037 l_ (r(1)li (4) Step 4: I 0. 0263; l*(r) 0.05454. li ( r (1)) = Il6(r )I (1) Since - < lc*(r i )l1,I go to Step (5). (5) Step 5: From (4) it is seen that components 1e2I = le2 = 2 5 0. 05454, sothattheset{e* (r(1)%= =C, (6) Step 6: m=2, and 2n+l=3. Since m < 2n+1, go to Step 7. (7) Step 7: The reference subspace U1 (i.e., U, since m= 2) containing e2 and e is the 2-dimensional subspace defined by the mapping i1:'U U m-, where 00'O O O 0 0 0 0 0 o o

191 (8) Step 8 yields e....) = [1m3 (r) = To 05454 0. 05454j O.4450 f() = [Il]Tf =.1 s ^ 0.1110 To determine [A( )(r)] begin with the matrix YQ 0000000 0 0 0 0 0 0 0 0 0 0 10 0 0 0 0 0 0 r r r0 0 0 00 0 0 0 0 0 0 0 O OOOrl r0 0 [R(r)] = 0 0 0 rl r0 0 0 0 0 0 0 0 r1 r0 0 0 0 0 0 0 0 r r0 0 0 0 0 0 0 0 rl ro 0 0 0 0 0 0 rl Find the matrix [Rl(r)] which is equivalent to [R(r)] according to Eq. C. 20 of Appendix C. This yields

192 010 00000 0 r 0 0 0 0 0 00 0 00 rl r0 0 0 rl' -O rO R( rO ~ ~1 [RI(r)] = |0 O0 r r 0 0~ ~ = 01 O O, 0 r0 0 0(1) r I 0:0rr00 O [ O O O O O rl r [A(l()] = [R()] = [ (9) Step 9: 1 2 (10) Step 10: Let -0. 053 ()(r)(2)) 1 and substituting it into Eq. 5. 10 when k= 1, m= 1, and where the values of f() and [A (r)] ( are those found in (8). This yields l er(2)

193 -0. 053 r 3 +0. 053 = 0.445 r 3 + 0. O O Solving this equation for r one obtains (2) -O. 48835 r(2) = L 1.00000_ (11) Step 12: Using the values obtained in (10) and the prescribed 9 9 values of f e U, the unknown components e e U can be determined from Eq. 5. 5 or equivalently from Eq. 5. 5a. For example, knowing the set of values f: i = 1,..,9 e 2 and r) r 1 then the unknown components e, %, C 6, 7, 8, E9 can be obtained from Eq. 5. 5a as follows: r o et= f + (fi -1 i1) 2, i= 3, 4, 6, 7, 8,9 Solving for the unknown i e. yields the vector -0. 01976 -0.053 0.0068 2) 0. 04123 (r ) = 0. 0530 0. 05337 0, 04866 0. 04264 0. 03669

194 (12) Step 13: Since lle(r ())11 0.05337 > 1(1) (r(2))1 = 0 053 go to Step 15. (13) Step 15: Form the set e, c. (14) Step 16: Since m = 3 = 2n+ 1 go to Step 19. (15) Step 19: The reference subspace U2n+, containing the components 12, e5 e6, is defined by the mapping 12: - U23, where O O O 0 0 0 100 000 000 01 0 0 0 0 001 000 000 000 (16) Step 20 yields ~(2) T O ~0.4450 f(2) = [2]T f = 0.1110 0817 (2) +1 and 3 O [A(2)(r)] = r r0 0 rl

195 Fr7] (2) S (2) (17) Step 21: Let r = lo, and let c (r) = p a Substituting it into Eq. 5. 10, when k= 2, and where the value of the vector f (2) (2) (2) a(2), and [A (2)(r)], are given in (16), yields p(-r o + 1) = 0.445 r + 0. 1110 0 0 p(r + 1) = 0. 111 r +0.0817 Solving for r and p yields, VO. 4847 - I r= 1.0000 p = 0.0541 (18) Step 22: Solving Eq. 5.5, whenr = r, 2 =-0.0541, 5= 0. 0541, and e 6 =0 0 541, yields -0.0296 -0. 0541 0.0081 0. 0426 (r) = 0.0541 0. 0541 0.0491 0.0429 0. 0373

196 Remark: One could have skipped (17) and (18) and have processed directly to (19) (i. e., Step 23 of the algorithm), since it is self-evident that the value of lle(r)ll < 0. 05454. Clearly, if the test of Step 23 indicates that r = r*, i. e, that the value of I1 E(r) II is the minimum possible value of lle(r)l II, then one must return to Steps 21 and 22 of the algorithm and calculate r* and (r*), respectively. (19) Step 23: Consider the equation 3 0 o -771 3 3 r 0 0.4450 r I 0 r] 0 10r. 081 L0; rO nj [7j [0 rO ri L.0817i Solve for {r ():v=l,2,3}, whenr (V)=1 01 1 7i = 0. 052 for i / v, and 77i =p() for i = v, where i=1,2,3. The results are given in Table 5. 1.

197 v 1 2 3 r -0.5035 -0 4944 -0.4917 O r 1.0000 1.00 10000 1.0000 1 0.08234 -0.0052 e2 -0.01694 -0.052 -0.052 e3 0.01746 0.00426 64 0.0428 0.0385 4 65 0.052 0.05094 0.052 66 0.052 0.052 0.05253 e7 0.04755 0.0478 68 0.04084 0.04213 e6 0.03569 0.03639 Table 5. 1. Results of Step 19 of Example 5. 1. (20) Step 24: From Table 5. 1 it is seen that r found in (17) is not optimum, because Ip( )I < Ip when v = 1, 2. (1) (2) (21) Step 25: The two vectors e(r 1) and e(r() are given in Columns 1 and 2 of Table 5. 1, respectively. (22) Step 26: From Column 1 of Table 5. 1, it is noted that when r=(, then le1 > 0.052. Hence, returntoStep 15 ofthealgorithm r=_, then Ie using the vector e(1)). (23) Step 15 yields the set e1, E, e6. Since m=3, proceed to -Step 19. (24) Step 19: The reference subspace U3 contains the components 61', 6e el.

198 (25) Step 20 yields (s. 0000 L+ 1 a (3) 1. (3)oo 0.0817 1+ 4O 0 (3) [A\ (r)] = -r14 r 0 rl 0 (26) Step 21: Let r = [r0 1]T and let e(3) = p c(3), then [A(3)(r)]T e(3) [(3)) T f(3) yields [A (r)] =[A (_r)] f yields p(r4- 1) = 1.00 r04-0.111 p(r+ 1) = 0.111r0 +0.0817 Solving for r0 and p, yields lo. 49826 r 1.00000 p = 0.0526 0. 0526 (27) Step 22: Solving Eq. 5.5, whenr =r, and when(3)(r) = 00526 L0.0526_

199 0.0526 -0 02705 0. 0098 0. 0428 e(r) 0. 0526 0.0526 0. 048 0. 0422 (28) Step 23: Solve the equation r -r 0 r -r 0 r ] 2E r r03 0 0817_ for r:v=1,2,3, when r = 1, 1 = 0 052 for i v v and 77i =P( ) for i = v where i= 1, 2, 3. The results are tabulated in Table 5.2. 1 (29) Step 24: Clearly,from Table 5,2, r found in (26) is not optimum. A(2) (30) Step 25 through 27: Using E(r (2) repeat Step 5. (31) Step 6: Since m= 2 < 2n+l = 3 proceed to Step 7. (33) Step 7 yields the reference subspace U42 containing the set 10 e tt in T e 5r(1) is ir(1) 0Note that in Table 5. 2 the vector r is identical to r of Table 5. 1.

200 v 1 2 3 (v) | "O( | -0. 5035 -0. 50025 -0.49948 ro o 1.0000 1. 00000 1. 00000 e1 0. 08234 0.052 0.052 e 2 -0.02924 e3 0.01276 e 4 0.04132 65 0.052 0.05163 0.052 E6 0. 052 0.052 0.05223 e7 0.04764 e8 0.04197 e9 0.03628 Table 5. 2. Results of Step 28 of Example 5. 1. (34) Step 8 yields,(4) 1. 0000 (4) + 1 f = 0 0817_' =, and 0 [A (r)] = (35) Step9: Take r(3) =- [ (36) Step 10: Let (4(r(3)) = 0.05 [a(4)] = andsubstituting it into Eq. 5. 10, when f( and [ (r)] have the values found in (34), yields

201 0.05 (r05 +1) = 1. 000 r05 + 0. 0817 Solving this equation for r0, yields (3) _= [ 50661] _ L 1.0000 (37) Step 12 yields 0.05 -0.03628 0.00618 (3) 0. 03648 e (r ) = 0.04843 0.05 0.0464 0. 04126 0. 03588 (38) Since Step 13 yields a "yes", Step 14 is next. (39) Step 14: Form the set {e1 E2, e6} from the components of e(r (3). The component e2 was added to the previous set since IA2I = max IAc.I, where IAE.l A li( )(r ())I and the 1<i<9 i' (3) (2) value of (r (3) and (r ) are given in (37) and in Table 5. 2, respectively. (40) Since Step 16 yields a "yes", Step 19 is next. 2n+l (41) Step 19: The reference subspace U5 contains the components {E21 2' C6 of E in U9 (42) Step 20 yields f() =[1.000] _ (5) f(5) 0.44501 = L, and n.nQd~0817_r ( i and

202 r0 0 [A((r)] = r 0 -r 1 T and let (5) (5) (43) Step 21: Let r = [ro 1] and let = pthen [A(5)T (5) [(5) T (5) yields p(r- 1) = 1.00 r0 +0.445 p(-r4 - 1) = 0. 445 r0 -. 0817 -0. 516 Solving for r andp, yields r = 0 p = 0. 046831 (44) Step 22 yields 0. 04683 -0. 04683 -0. 00381 0.02904 c(r) = 0.04342 0. 04683 0. 04451 0.04012 0. 0353 (45) Step 23: Solve the equation 0 ] -E1] = 0.4450 L0 1'. r - 0.080r

203 for r(: v= 1,2, 6 whenr() =1 and e= 0.045 for i v, ~ i }) when r (V)_(. = p() for i = v, where i= 1, 2, 6. The results are given in Table 5.3. v 1 2 6 r) -0. 523 -0. 5208 -0. 5131 (V) Yr' 1. 000 1.0000 1. 0000 e 00.06312 0.045 0.045 e2 -0.045 -0.0524 -0.045 6 60 0.045 0.045 0.04773 Table 5. 3. Results of Step 45 of Example 5. 1. (46) Step 24: Clearly from Table 5.3, r =r* -= 0 5. Therefore, e** = e(r) found in (44). (47) Step 31: Eq. 5. 11 becomes -0.516 + z = 0 Therefore, z* = 0. 516. (48) Steps 32 and 33: Using the first component of Eq. 5. 13 when z = z* found in (47) and when e(1, z) = -** found in (44), one obtains 1.000 = * +0.04683 Hence, 3** = 0. 95317. Therefore, the optimum parameter pair for the Chebyshev approximation problem is given by

204 (3**,z*) = (0.95317, 0.516) Furthermore, the best Chebyshev approximating vector, f**, to 1.0000 0.9 5317 0.4450 0.49183 0.2500 0.25381 0. 1600 0. 13096 f = 0.1110, is given byf** = 0.06758 0.0817 0.03487 0.0625 0.01799 0.0494 0.00928 0.0400 0.00470 where 0.04683 -0.04683 -0.00381 0.02904 c** = 0.04342 0. 04683 0.04451 0.04012 0.03530 and where li**lII 0.04683. -- Co Figure 6 depicts graphically the components of the final error vector e**. For comparison, the initial error vector, C*(r ), (i. e., Ruston's final error vector) is also shown. In Fig. 7, we have attempted to illustrate graphically the results of using the method of descent in minimizing the function 11e(r) 11. The numbers in parentheses refer to the steps of the iteration used in the above numerical example. to the steps of the iteration used in the above numerical example.

205 06 o 04. 02 -. 0E4 -.06 Fig. 6. The Chebyshev approximation error, optimum approximation yields 11** = 0. 0468 Ruston's approximation yields *(z(1))O = 0. 0545'~~~~~~~~~~~~0

(45) 0,.60 0,.58 (3) 0,56 \ ^^18) ^0.54 (45) 0 (\/ 27) - 0.52 0. 50 3 7) ^ ^^ (42) 0. 048 (44) 0,046 -0.53 -0.52 -0.51 -0,.50 -0,.49 -0.48 Fig. 7. lie(r)II vs. r for Example 5. 1. 0 - co0

207 Example 5. 2: Consider again the problem of approximating the vector f given in the previous example, but this time let n= 2. (1) Step 1: Eq. 5.3 becomes 1. 0000 r 1 1 el 0. 4450 z1 2 PI 2 0. 2500 z 2 z2 E 1 2 3 0. 16001 z4 23 e 0. 1110 = z z + 5 1 2 5 0. 0817 z1 z2 6 0.0625 z 6 z26 1 2 7 0. 0494 z z2 8 0.0400) Z1 Z281 9 (5. 14) Note that q= 9 and n= 2. (2) Step 2: Eq. 5. 7 becomes 1. 000 0-.4450 0.2500 rO 0. 4450 0. 2500 0. 1600 0.2500 0.1600 0.1110 r 6(r) = 0. 1600 0. 1110 0.0817 0. 1110 0. 0817 0.0625 r 0.0817 0.0625 0.0494 0.0625 0.0494 0.0400 Let r() denote the vector r having the j-th component equal to one. Minimizing 1I6(rj)ll with respect to r(, where j = 1, 2, 3, yields the Minimizing:lI6(r 00

208 results given in Table 5. 4. j 1 2 3 r) 1 -0. 20092 0.20331 r ()-4. 976793 I -1012 r) 1 4.911880 -0.98695 1 1(iJ) 0. 013297 -0. 0026718 0.0027072 62(j | -0 013297 0.0026718 -0.0027072 63(j) O -0.001068 0.0002148 -0.000217 (6J) 0. 008877 -0.0017835 0. 0018072 4 5(j) | 0.001139 -0.0022883 0.0023186'(J) 0.013297 -0.0026718 0.0027072 6 - <j 0.013122 -0.0026365 0.0026714 0.00122121 0. 00122118 0.00122121 Table 5. 4. Results of Step 2 of Example 5. 2.

209 From Table 5. 4 it is seen that all the vectors {r()} give the same minimal value of 11 6(r) 11 in the set 11 rl I = 1. We select for the initial 00 estimate the vector, r3, namely, 0.20331 r(1) -1.0128 1.0000 The corresponding value of ll (r(l))ll0 = 0.0027072. 0c (3) Step 3: From Eq. 5. 11, one gets z =(1) [ Substituting this value of z into Eq. 5. 14, yields the relation of Eq. 5. 4. The best Chebyshev approximation yields the error vector e*(r(1)), the value of which is given in Table 5. 5, Column I. li a(r(1) I oc (4) Step 4: = 0.00122, le *(r)) ) = 0.00656. llr~ll^ - 1c 116 1 Since (1 < llell, go to Step 5. - 1 (5) Step 5: From Column I of Table 5. 5we obtain {e 1,, e9. (6) Step 6: m=3, and 2n+1= 5. Since m < 2n+l, go to Step 7. 3 (7) Steps 7 and 8: Form the reference subspace U1 containing l, e4, eC. Hence, f(l), (r()), and [A()(r)] in U1 are:

210 I II r ().20331 0.20316 0 r(j) - 1.0128 -1.0128 r2() 1.0000 1.0000 e1 0.00656 0.0065 e2 -0.00424 -0.00591 e -0.00504 -0.004846 j 3 E4 -0.00656 -0.0065 e5 -0.00584 -0.00586 5 6 -0.00277 -0.002826 67 0.00070 0.000632 e8 0.00399 0.003913 e9 0.00656 0.0065 Table 5. 5. Values of i(r) for Steps 3 and 10 of Example 5. 2. 1.00 0.00656 f(1) = 16, E (r)) -0.00656, and 00. 04 _0.00656

211 3 4 5 2 4 2 (r0 1 O0 r 2 -3r0 r1 r2) 7 5 2 3 2 3 3 [A (r)] (r 6r 0r r+ 10 r r 2 r 4r0 r r2 2 5 6 (r1 r2 - r0 2) r (8) Step 9: Letr(2) -1.0128 1. 0000 (1) (2) 0. 0065 (9) Step 10: Let c ((r() = 0 0065 0. 0065 Solve Eq. 5. 10 for r0, when k= 1, m= 1 and using the values of f( and [A (1(r)] (2) found in (7). This yields r=r 0.20316 (2) 1. 0128 1. 0000_ (10) Steps 12 and 13: Solve Eq. 5.5 for e(r( ) U9 using r=r( (1) (2) and the vector ((r(2). The values of the components of the resulting (2) (2) vector E(r2) are given in Table 5.5, Column II. Note that ll (r) 11 = ll(1) (r(2)) 11 (11) Step 14: Form the set { 1, E2,e4' 5', 9. The component 2 is chosen because A le 2 = max IAe I, where A i is the differ1<i<9 ence between the values of i-th components given in Columns I and II of

212 1l (2) Table 5. 5. The component 5 is chosen because E (r() > lec(r( )1. (12) Step 16: Since m= 5= 2n+l= 5, go to Step 19. 5 (13) Steps 19 and 20: Form the reference subspace U25 containing the set of components e 1', 2 e4 e5 e91 Hence, f(2) (2) and (2) 5 [A (r)] in U2 are: 1.000 +1 0.445 - f(2) 160(2)(2) f = 0.160 a_ -I 0.111 -I 0.040 I r rr 0 0 (rl -ror2) r2 0 [A2)(r)] -r22 (r0r2-r12) (2 rlr2-rOr 1 ) 0 -rlr2 0r2 r r 2 0 0. 2 r0 (14) Let _( = rJ and _(r) = p a. Then, according to Step 21, solve for {rO, r1, p from the relation, Note that we have skipped an iteration step by selecting E 5 here. If this results in an(r) e Uq with lE(r) II > le(2)(r) II, where (2)U 5 then we can always return and add the missing step.

213 r0r1 (r1-0) -1 0 l.OOOp 0.445 +p O r02 (ro-0r-) -r1 0 45+p 0 ~r0 -r 1O0.160+p =0 o 0O (2r02rl-r0r13) (3r0r12 -14-r 0) 1 Oll0p 0,040- p This yields that r0 = 0. 20921, r1 = -1. 02457, and p = 0.005628. (3) (15) Step 22: Usingr(3) andp found in (14), Eq. 5. 5 yields the vector (3) E (r ) tabulated in Column I of Table 5. 6. (16) Test r(3) to see if it is the optimum vector r*, according to Steps 23 through 27. The results are given in Columns II, III, IV, V, (3) and VI of Table 5. 6. From these results it is seen that r is not the optimum (see Column IV of Table 5. 6). (17) Steps 5 and 8: From Column IV of Table 5. 6, form a new subspace U4 which contains the set of components {e', e2 c, 6 9 of e. Hence, f(3)(3) and [A(3)r)] in U3 are: 1.000 +1 f(3) 0. 44511 (3) - 1 0.040 + 1_ (r0r1 - r r2) 0 = (r1 2rr r2) (ro3r13 - 2r04 r1r2) r23 (rl6 - 5rOr14 r2+ 6rO2r2r2 -r 3r23) 0 (rOr25 - r2r2) [A(3()]t ^~~~ ~~~ 2r~ oS la

I II I l IV V VI VII r0.20921 0. 212997 0. 212004 0. 21036 0. 209907 0.209907 0. 218972 r1 -1. 02457 -1. 03046 -1. 02907 -1.02676 -1. 02602 -1. 02396 11.043247 I r2 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1 el 0.005628 0,.011717 0,0055 0.0055 0.0055 10,0055 10.0045 e 2 -0.005628 -0.0055 -0.006547 -0. 0055055 0. 055 -0.0055 -0.0045 63 -0.003670 -- -0. 003353 -- 0.000957 E4 -0.005628 -0.005 5 5 -0.005366 -0.005 5 -0,0055 -0.003385 e5 |-0.005628 -0.0055 -0.0055 -0.0055 -0.005601 -0.0055 -0.0045 E 6 -0.003143 -- - -0.00313 -- ~ - -0.003019 67 -0.000028 -- -- 0.000094 -- -0.000593 g 8 0.003085 1 ~ 0.002975 -- -- 0.002129 e9 0.005628 0.0055 0.0055 0.0055 0.0055 0.005723 0.0045 Table 5. 6. Results of Steps (14) through (16) of Example 5. 2.

215 (4) (3) (4) (18) Steps 9, 10, and 12: Let = r r, and let (r) = (3) 0.0045 a(3) Then, solving Eq. 5.10, when k= 3, yields the results given in Column VII of Table 5. 6. The values of the components of (4) 9 c(r(4)) U9 have been obtained from Eq. 5. 5. (19) Since 11e(r(4))l = l11(3)E(4))i, thenaccording to Step 13, calculate I A I using the values of the ei's given in Columns VII and IV of Table 5.6. From the set of IAei, it is seen that l Ae3 = 5 max l Ae. I. Hence, we form a new reference subspace U which 1 4 1<i<9 will contain {el e2, e36,, e9} (20) Step 20: f (4) a (4)and [A(4)r)] in U5 are as follows: 1.000 -+1 0. 445 ( (4) (4) f(4) 0.250; a = +1 and 0.111....0. 040 +1 "0 - r —0 0 rI r 0 [A (r)] r2 (r r2) r1a- 2r0r2) 2n 1 2) 1 0 1 2 2) 10 -r2 (-r2 +4rOr r r 2 - 0 0 r24

216 r0 (21) Steps 21 and 22: Let r() r and ((r()) = p(4) 1L Then, solving Eq. 5. 10, when k= 4 and r = r(5) yields the results tabulated in Column I of Table 5.7. (5) (22) Test this vector r to see if it s the optimum vector r* according to Step 23. The results are given in Columns II, III, IV, V, and VI of Table 5.7. From these results it is seen that r(5) = r* Therefore, 0. 23212 r* = 1.06873 1.0 (23) Step 31: Using the value of r* in (22), determine the vector z* from Eq. 5. 11. This yields 0.76552 z* O.303 17 (24) Substitute this value of z* and the value of E** given in Column I of Table 5.7 into Eq. 5. 14. Then, determine P** from the first two components of this vector relation, namely 00ooo0 1.00000 1.ooooo 0i *l 0. 00284 0.445 JL0.76552 0.30317 1 * -0. 00284

I II III IV v VI r0 0.232118 0.233236 0.232803 0.232408 0. 2319098 0. 231352 r1 -1.068733 -1. 070449 -1. 069855 -1. 069302 -1.068469 -1. 067228 r2 1.00 1.00 1.00 1.00 1.00 1.00 0.0028387 0.0046685 0.0028 0.0028 0.0028 0.0028 62 0-0.0028387 -0.0028 -0.0030519 -0.0028 -0.0028 -0.0028 63 0.0028387 0.0028 0.0028 0.0029247 0.0028 0.0028 3~~~~~~~~~~~~~~~ 64 -0.0001983 65 -0.0028387 -0.0028 -0.0028 -0.0028 -0.0029222 -0.0028 66 -0.0027783 -- -- 67 -0.0013608 -- - e8 0.0007587 -- -- -- -- -- E9 0.0028387 0.0028 0.0028 0.0028 0.0028 0.0029934 Table 5. 7. Results of Steps 21 and 22 of Example 5. 2. Table 5. 7. Results of Steps 21 and 22 of Example 5, 2.

218 Hence, 0.31476,**= =: _. 68240 Therefore, the optimum vector pair is, r0.31476 F0.76552 \ \0.68240 L0.30317 / and the best Chebyshev approximating vector to the prescribed vectors f is given by 0.9972 0.4478 0.2472 0.1602 f** [Z(z*)] ** 0.1138 -.0. 0845 0. 0639 0. 0486 0.0372 The resulting Chebyshev error vector is given by 0.0028387 -0.0028387 0. 0028387 - 0.0001983 e** = -0.0028387 -0. 0027783 -0. 0013608 0.0007 587 0.0028387

219 and II e** = 0.0028387. -- oC Figure 8 compares the components of the final error vector, e*, and those of the initial error vector, E*(r()). The improvement in the approximation is evident and it will be discussed in Section 5. 5. Example 5. 3 Consider the problem of approximating the real vector 2.8 1 2 L2 Zlz2 22 f = 2.0 by the real vector f(, z) = z1 z2 14 2 1. 0 z Iz2 so that Hlf - f(3** z*) I! is minimum for all ( z) e z x. (1) Step 1: The following vector relation in U must be considered 0 1 1 31 1 01 2.8 Z1 z2 L2 e2 2 = z1 z2 + (5.15) 1.ot0 that 3 3 1-0, zI z2 E. 0Noz 4 4 1.01. 5 Note that q= 5 and n= 2.

220 0. 006 /./to~~~~~~~ / -0.004 / -0. 006 \E/ Fig. 8. Chebyshev approximation error, optimum approximation yields lE ** 10 = 0. 00284 Ruston's approximation yields 11E*(z( ))II = 0. 00656

221 (2) The minimal value of l1&(r) II in the set lIrl = 1, was obtained in Example 4. 1 of Chapter IV. The vector r() is taken to be the (3) vector r of Table 4. 1, namely 0.04098 r(1) = -0. 60109 1r and the corresponding 11611 = 0.3169. oC (3) Step 3: (1) F ~0.662916 z I -0. 061823 The best Chebyshev error vector e*(r(1)) e U5 is given in Column I of Table 5. 8, (4) Step 4: Since I I(1) () = 0.193 < lle*(r (l)ll = 0.20315, go toStep 5. - 1 (5) Steps 5 through 8: Form the reference subspace U13 containing the set of components {el' e4, e5 Hence f(), ()r(1)) and [A(1r)] 3 in U1 are:

222 I II lr -0.04098 -0.04425 r1 - 0.60109 -0.60109 r2 1.0 1,0 E -0.203152 -0.2 e2 -0.17827 -0.163 C3 0.201457 0.2217 E -0.203152 -0.2 4 E5 0.203152 0.2 Table 5. 8. Results of Steps 3 and 6 of Example 5. 3. 0 -0.203152 f(1 = 1, e( 1)(r(1)) = -0.203152 0.203152 3 r0 A (r) = (r1 2r0r1r2) (rr2-rr2) ri 0 21 (6) Steps 9 and 10: Let (2)= 0.60109 andlet (1)(r(2)) -0.2 (3) Substituting into Eq. 5. 10, when k= 1 and r = r yields

223 r0 + 3.213116 r + 0.142156 = 0 Therefore, r0 = -0.04425 (2) (2)\ The value of r2 and the corresponding value of E(r(2)) U is given in Column II of Table 5. 8. (7) Since lle(r(2))II > llk1(r2) I go to Step 15 and from the set {c1' 3, 4, 5}. (8) Since m= 4 < 2n+l= 5 go to Step 18 and form the reference subspace U24 containing the components e e1' c3, e4, e 5} (9) f(2), e(2)(r(2)) and[A(2)(r) in U24 (9), (r and A r) in U2 are: 0 -0.2 f(2) _ 2 _(2)(r(2)) 0. 2217 1 -0.2 l_ 0.2 r 0 (r 0- r, ro [A(2)(r)]) r -r r2 0 -..O.. 2

224'r -0. 19 (10) Let (3)= andlet (2)(3) 0. 19 -0. 19 1 L 0. 19 (3) Substituting into Eq. 5. 10, when k = 2 and r r (3) yields 0.19r0 + 1.81r0 1. 81 r 1.19 = 0 0 0 1 1.19r11O 1.81 r + 1.19 r + 0.,81 = 0 Solving for r0 and r1 yields that both r0 and rI are complex. Therefore the above equations have no real solution. Go to Step 28. (11) Steps 28 and 29: The vector f(2) U24 and the matrix [A(2)(r)] are given in (9). The vector (2) e U24 is (2) + I +1 Hence Eq. 5. 10, when k = 2, r = (3) and (2)(r) = p (2) becomes 2(r0- r12) - rr = p(-r02+ r- r1 + r1) 2r0 + r 1 = p(r r + 1) Solving for r0 and r, which yields the minimal value of IP, one obtains1 r = 0, r1 = -2/3, and p = 0.2. The vector r(3) and 12See Example 4.3 of Chapter IV.

225 the corresponding value of (r(3)) U5 is given in Column I of Table 5. 9. (3) I(3) (12) Step 30: Since lil(r3))1 = IPI, then(3) = *, namely 0 r* =-2 3 (13) Let us now deviate from the procedure and determine the 5 solution vector r when the (2n+1) components of c e U are equal in absolute value. Hence, let us solve 0 r0 r 1 0 0.r r 1 0 r0 1 1 0 12 = 0 r r 1 0 pa 0 0 r0 r I 0 r rl 1 1 01 for r0, rl, andp, when +1 - a = +1, or a = +1 - +1 The solutions which yield the minimal value of Ie Ii are given in Columns II and III of Table 5. 9. It is noted that the resulting value

I II III IV V V I VII r 0 0.018717 -0.0447 -0.4475 1 0 0.888 1 -2 -0.6937 -0.5951 0 0 -0.669 -2.433 r3 1 1 1 -0.7846 1 1 61 -0. 2 -0. 20043 -0.20346 -4. 045 -0.19 -0.M19 -0.19 62 0.1 0.20043 -0.20346 0.141 1.364 1.589 -0.819 6Q 0. 2 0. 20043 0. 20346 0. 19 1.866 0.19 0. 19'3 E. -0.2 -0.20043 -0.20346 -0.19 -0.19 -0.211 -0.19 74 -~ o0. 2 0. 20043 0.20346 0. 19 0.19 0. 19 -0. 288 Table 5. 9. Results of Steps 11 and 13 of Example 5. 3. Table 5. 9. Results of Steps II and 13 of Example: 5. 3.

227 of {I i [ is greater than the value of e lio given in Column I of Table 5. 9, the case when 2n components of E are equal in absolute value. As a further digression, let us test if there exists an r which (3) yields a lower value of i E(r) {{ than 1e (r ), when (2n- 1) components of (r) 6 uq are equal in absolute value. Following the procedure given in Step 23 of the algorithm, we decrease the values of each subset of 3-components out of the set {e6, e3, 4 e5 } and solve the resulting relations for r. The results are given in Columns IV, V, VI, and VII of Table 5. 9. It should be noted that when performing this test, one must solve an under determined equation. For example, when 63 -64= = =0. 19, i. e., the case which yields the values given in Column IV of Table 5.9, one obtains the relation 1.81rr + 1. 19 r + 0.81 = 0 (5.16) where r0 and r1 are unknown. Although this equation can be solved, by arbitrarily selecting one of the unknowns, one is interested in the solution which minimizes the value of e(r)l. We have assumed 00 (without proof) that such a solution occurs when the value of J el is minimum. Hence, we have solved Eq. 5. 16 subject to the constraint 12 = minimum. Note that the component e1 is related to the prescribed components {e3, 4, e^ } by

228 (f3- 3)(r- r1 ) + (f4- E4) r ~1 - 0 r' 0 1.81 (r0 r1 ) + 1.19 r ra 0 By using the relation between r0 and r, given by Eq. 5.16, we can obtain the component E1 as a function of r (or r1), and the r0 for which c12 = minimum, is obtained from dE1 (r0) dr = The resulting values of r0, r1, and E1 are given in Column IV of Table 5. 9. From Table 5. 9 we note that the vector r* obtained in (12) yields the minimal value of E {(r)l{. 0 (14) Step 31 yields z* = -2/. (15) Step 32: Substituting the value of z* in Eq. 5. 15 yields 0 I1 1 -0.2 2. 8 0 0. 6667 L2 0. 2 = 0 0. 4444 + 0. 2 1 0 0.2963 -0.2. 0O 0. 1975 0. 2

229 Solving for 1, and i2 yields 3.85 L** = (5. 17) L4.05 It should be noted that the component a1** may take on any value in the interval [-4. 25, -3. 85] without affecting the value of 1 e*l = 0. 2. - C Clearly then, the best Chebyshev parameter vector pair (0**, z*) zB x x is not unique. (16) The optimum vector pair is / -3. 85 0 ** z*)= \ 4.05 0. 6667 The optimum Chebyshev approximating vector and error vector are: 0.2 -0.2 2.7 0. 1 f** 1., and e** =.2 1.2 -0.2 LO. o8 0.2 Example 5. 4 Consider the problem of approximating the function f(t) = (4 - t - 0. 1 cos irt) by the set of n 2 exponential functions in the finite point set Te = {0,1,2,3,4}

230 (1) Step 1: Given q= 5, n=2, and 3.9 3.1 f = 1.9 -0. I (2) Step 2: The minimal value of 116(r) 11 in the set llr ll = 1, was obtained in Example 4. 1 of Chapter IV. The minimal value of 116(r) I = 0. 4, which is obtained when r is given by r.2 L1 (3) Step 3: The corresponding vector z(, obtained from Eq. 5. 11, is * Z(1) = 0 z =[ Since the two components of z( are equal, the matrix [Z(z)] is given by Eq. 1. 18b (Chapter I). This problem was considered in Example 4. 1 of Chapter IV. In this example let us assume that the matrix [ Z(z)] is given by Eq. 1. 8 so that we shall consider the relation

231 3.9 1 1F1 c 3.1 1 I i E2 1.9 = + -0.1 1 1 - The best Chebyshev parameter vector 1* is found to be 1.9 and the corresponding error vector e*(r(1)) is given in Column I of Table 5. 10. I 11 111 IV V VI r0 1 0.933 1 1.02 1.016 0.8081 r1 -2 -2 -2 -2 -1.986 -1.6903 r2 1 1 1 1 1 1 E. 2. 0 0.9 0. 4 0.2 0.15 1 E2 1.2 0.9 0.4 0.2 0.15 0.1 E3 0 0.3 0 -0.126 -0.15 -0.1 64 -0-0.8 -0.047 0 0.006 0. 0251 0. 144 E 5 -2.0 -0.9 -0.4 -0.2 -0. 15 -0. 1 Table 5. 10. Results of Steps 4 and 5 of Example 5. 4.

232 11 00 (4) Step 4: Since ir1) = 0.1 < llc*(r )ll = 2.0, goto — ~ O0 Step 5. (5) The results of the different steps pf the procedure are given in Columns II through VI of Table 5. 10.'r0 (6) Steps 21 and 22: Taking r = rl and E(r) = p where +1 __ = - + 1 then the solution of Eq. 5. 10 (namely, Eq. 5. 5 since q= 2n+l) yields the results given in Column I of Table 5. 11. (7) Testing if this vector r is the optimum r* according to Step 23 yields the results given in Columns II through VI of Table 5. 11. From these results it is seen that -0o.1 +0.1 r = -2, and E_(r*) = -0. 1 _1, +0.1 0. I

233 I II InII IV V VI rO 1 1 1 1 0.9269 1 r1 -2 -1.9615 -1.9612 -2.016 -1.8898 -2. 130 r2 1 0.8845 0.9227 1.015 1 1.217 E1 -0.1 -0.2438 -0.09 -0.09 -0.09 -0.09 62 0.1 0.09 0. 129 0.09 0. 09 0.09 e3 -0.1 -0.09 -0.09 -0.15 -0.09 -0.09 e4 0.1 0.09 0.09 0.09 0. 129 0. 09 4 e5 — 0.1 -0.09 -0.09 -0.09 -0.09 -0.233 Table 5. 11. Results of Steps 6 and 7 of Example 5. 4. It is noted that this result is identical to that given in Example 4. 1 of Chapter IV. Therefore, the matrix [Z(z*)] is given by 1 0 1 [Z(z*)] = 1 2 13 1 4 and the vector **, found from Eq. 5. 3, is A~ F4

234 13 It can be shown that the best Chebyshev vector pair (p** z*) = 1 l),- ^i ^J 1 yields the vector pair (a ** *) Thus, the approximating function f**(t) = (a** + a2** t) e becomes f**(t) 4- t It should be noted that since the error function e(t) ^ f(t)- f**(t) = -0. 1 cos t then the approximating function f**(t) represents the best Chebyshev approximating function for a larger point set, i. e., for the point set See Section 6. 2 of Chapter VI.

235 Te = {t(i-1):i=1,2,.., q; q> 2n Example 5. 5 Consider the problem of approximating the function f(t) = t e, in the finite point set Te = { 0, 1, 2, 3, 4 }, by the set of n-exponential function where (a) n= 1, and (b) n=2. Hence, we have q= 5, and 0.0 0.36788 f = 0.27067 0. 14936 0. 07326 (a) When n= 1, the vector r is defined by rO r = Equation 5.7 becomes 0o.0 0. 36788 6() - 0.36788 0.27067 r0 B - 0. 27067 0.14936 O. 14936 0. 07326_J The vector r which yields the minimum value of 11 6(r) II in the set llr I1 = 1, is found to be r- 1.7358 r(1) I

236 (1) and the corresponding value of 11 6(r )11 = 0.36788. Equation 5. 11 becomes P(z) = z- 1.7358 = 0 so that z) = 1. 7358. Equation 5. 4 becomes 0.0 1 f 6E 0.36788 1.7358 1E 0.27067 = 3. 0129 + 6 0.14936 5.2296 64 0.07326 9.0773 e5 The ft which yields the minimal value of II II is found to be i* = 0. 040797. 00 The corresponding value of e* is given in Column I of Table 5. 12. Since 1(r 1)) 11e* 11 = 0.2971 > (1)o = 0.13447 a: (1) we minimize lle*(r) II according to the algorithm. The results of the different steps of the procedure are given in Columns II and III of Table 5. 12. The optimum vector r* is found to be -1. 0624 r* 1

I II III IV V VI VII r0 -1.7358 -1. 3997 -1. 1763 -1. 0624 -1.0392 -1.0988 -1.2993 r 1 1 1 1 1 1 1 -0.0408 -0.0842 -0.1427 -0.1673 -0.200 -0.16 -0.16 0.2971 0.25 0.2 0.1673 0.16 0.192 0.16 0.1478 0.1057 0.0732 0.0576 -- -- -- E4 -0.0640 -0.0816 -0.0829 -0.0771 -- -- -- 65 -0.2971 -0.25 -0. 2 -0.1673 -0.16 -0. 16 -0.38 Table 5. 12. The results of the minimization procedure

238 and the values of the components of the error vector e*(r*) are given in Column IV of Table 5. 12. The results of the test are shown in Columns V, VI and VII of Table 5. 12. Therefore, the best Chebyshev pair is (j**, z*) = (0. 1673, 1. 0624) and the corresponding pair (a**, s*) is (a**, s*) = (0. 1673, 0.0607) Thus, the approximating function f**(t) = a** et becomes 0. 0607 t f**(t) = 0.1673 e (b) When n= 2, the vector r is defined by r] r = r1 Equation 5.7 becomes Equation 5. 7 becomes 0.0 0.36788 0.27067 r 6(r) = 0.36788 0.27067 0.14936 r1 0.27067 0. 14936 0.07326 r2

239 The vector r) which yields the minimal value of 11 8(r) Il in the set 11 r = lis found to be 0. 13534 ri) = -0 73576 and the corresponding value of 1i16(r(1)lI = 0. Since ll6(r(1))ll = 0, (1) * the vector r(1) = r*. Equation 5. 11 becomes P(z) = z2 -0.73576 z + 0. 13534 = 0 the roots of which are given by 0. 36788 z* = 0.36788 Since the components of z* are identical the matrix [Z(z*)] is given by Eq. 1.18(b), i.e., 1.0 0.0 0.36788 0.36788 [Z(z*)] = 0.13534 0.27067 0.04979 0. 14936 0.01832 0.073226 Calculating the value of,** which satisfies f = [Z(z*)] f** yields _J**

240 Therefore, the best Chebyshev vector pair (**, z*) is given by P/ 0 0.36788 \ \ 1 0.36788 / It can be shown14 that this yields the vector pair a* =( ra L1 )= Therefore, the approximating function sl*t f**(t) = (a** + 2t) becomes f**(t) = t e which is identical to the prescribed function f(t). Example 5. 6 Consider the problem of approximating the function f(t) = (2e te t/), in the finite point set Te = 0, 1, 2,..., 6 by the set of n=3 exponential functions. Hence, we are given q= 7, n=3, and 4See Section 6. 2 of Chapter VI.

241 1 0. 12923 -0. 09721 f = -0. 12356 -0.09870 -0. 06861 -0. 04483 Since n= 3, the vector r is defined by r r = r r lr.2 33 and Eq. 5.7 becomes 1 0.12923 -0.09721 -0.12356 r0 0.12923 -0.09721 -0.12356 -0.09870 r 6(r) = IF]r 6() = [F = -0.09721 -0.12356 -0.09870 -0.06861 r -0.12356 -0.09870 -0. 06861 -0.04483 r Note that the 4x4-matrix [F] is of rank 2. Therefore, the minimal value of 11(r) II = 0, in the set llr II1 = 1. Furthermore, the vector r, which yields 116(r) II = 0, may be obtained by making one of the components of r equal to zero. The value of the various vectors r, thus obtained, are given in Table 5. 13, where the values of the corresponding vector pairs (3, z) are also given. Note that each set of vectors {r, z, i 3, given in Table 5. 13, represents the set of best Chebyshev

242 I II III IV r0 0.22313 0. 2174. 05109 0.0 r1 -0.97441 -0.7260 0 0.22313 r2 1 0 -0.7454 -0.97441 r3 0 1 1 1 Z1 0.36788 0. 36788 0.36788 0.36788 2 0. 60653 0. 60653 0. 60653 0. 60653 z3 -- -0.97441 -0.22899 0 31 2 2 2 2 I, 1-1 -1 -1 -1 P3 -- 0 0 0 3 3 Table 5. 13. The best Chebyshev parameter vectors of Example 5. 6. parameter vectors {r*, z*, P** since 1161 = 0implies lle**ll =0. It should be noted that this example illustrates the case when the ~..*r isr n+ 1 minimum value of le*(r) ll is attained at the point r e En with the,. *- - -;.-~.V - - -- component r = 0 (see Column I of Table 5. 13). As was mentioned in Chapter IV, in such a case, there exists another vector r e E, with r / 0 and which yields the same approximation. This is shown in n Columns II, III, and IV of Table 5. 13, where we note that although the n-th order polynomial equation contains an extra root, this root does not contribute to the approximating vector since /33 = 0. Therefore, we could have selected n= 2 rather than n= 3 at the start of the approximation.

243 Now before concluding this example, let us make some observations concerning the values of the extra root z3, given in Table 5. 13. First, note that { l' z2 represent the two roots of the polynomial equation P(z) = r3 +r2 z +r1 +r0 = 0 when r3 = 0. Recall that in the case when r3 # 0, the coefficients {r0 rp1 r2} are related to the roots of P(z)= Oby rg - Z z z r = (ZlZ2 + z2z3 + Z3) and r 2 -(Z1 + z2 + z 3) r3 where z3 denotes the extra root which does not contribute to the approximation. From these relations, one can obtain the value of z3 in terms of zl and z2 as follows: When r0= O, then Z3 = O0 when r = 0, then Z = - and When r-:0, then z+z when r2= 0, then Z = -(z, +z2)

244 It is observed that the values of z3, given in Columns II, II, and IV of Table 5. 13, satisfy these relations. 5. 5 Analysis of Examples The method of approximation used in this thesis represents an improvement over that method used by Ruston (Ref. 20). This improvement is summarized in Table 5. 14. Example Ruston's IIE ** Percent Number * o 0 Decrease 00 5.1 0. 05454 0.04683 14 percent 5.2 0.00656 0.00284 57 percent 5. 3 0.20315 0.2 1.5 percent 5. 4 2.0 0.1 95 percent 5. 5a 0.297 0. 167 44 percent 5.5b 0.0 0.0 -- 5.6 0.0 0.0 - Table 5. 14. Comparison between the initial and final values of IE II.... -.. - oc.:-. The percent of decrease, given in Table 5. 14, is defined by e *(r) 11 - ll e** lI.-' i~00 —. 0 Percent Decrease = - x 100 I *(r) llII The improvement achieved by increasing the dimension, n, of the

245 parameter spaces is also illustrated in Table 5. 14 by a comparison of the results of Example 5. 1 (where n= 1) and Example 5. 2 (where n= 2); or of the results of Example 5. 5a (where n= 1) and Example 5. 5b (where n= 2). Examples 5. 1, 5. 2, 5. 4, and 5. 5a illustrate the case when the best Chebyshev approximation is characterized by an error vector e** which has (2n+1)- components equal in absolute value to llc ** I. It is seen that the vector pair (3**, z*) e 9B x is unique. Furthermore, the signs of the (2n+l) components of E** with absolute values equal to IIe** II alternate (2n+l) times, i.e., sgne** = -sgn** v=,2,..., 2n+1 v+l v where the components ei* satisfy * l**= Ic**l, v=1,2,.. 2n+1 v Example 5.3 illustrates the case when the best Chebyshev approximation is characterized by an error vector E** which has 2n-components equal in absolute value to lIE** IIl. First, let us note that since the interval 1! (r )) I1 o( ~ 1I, le*(r 1))ll is small

246 then the initial estimate of r, given by r), may represent the vector r*. The second point to be noted is that since one of the components of the vector z* is equal to zero (or equivalently, the component r = 0, of the vector r), then the best Chebyshev approximation is not unique. Furthermore, if -4. 25 < 1** < -3.85, where 1** is the first component of the vector i** given by Eq. 5. 17, then the resulting error vector e** will have only (2n-1)-components equal in absolute value to II **ll I, without affecting the value of IIE ** I 0. 2. For example, if,1* = -4. 05, then the vector pair (f**, z*) = - 4. 05 0. ) \ 4. 05.0 667T yields the error vector 0.0 0. 1 e** = 0.2 0. 2 -0. 2 0.2 It should be mentioned that, in the examples considered above, the projections of the components of e** e Uq which are equal in absolute value to lle**llc, represent the best Chebyshev error vectors in at least n+1 reference subspaces U5 U: j= 1, 2,..., n+1. 15 the discussion relating to Eq. 4.87 of Section 4. 5, Chaper See the discussion relating to Eq. 4. 87 of Section 4. 5, Chapter IV

247 Specifically, in Example 5. 1, the projections of the components eI, ei2, e6 represent the best Chebyshev error vectors in the 2 2? 2 2-dimensional subspaces U^2, US, where U contains the compon2 1 u2 0 i nents E *,, and U contains the components E *,. In E 5.2 02 2 65 Example 5.2, the projections of the components 1* * 2,;*, * 65'I represent the best Chebyshev error vectors in the 3-dimensional sub3 3 3 3 spaces UO3, U3 U 3, where U6 contains the components ce*, 6 ^1 2 3\) 1 3 3 Ua contains the components e *, e*, 6*, and where U0 contains 2 ( 3 the components *, ** *9* In Example 5. 3, the projections of the components e**, e;*, e4, e represent the best Chebyshev 3 3 3) 3 error vectors in U, U0 3, U3 {, where U3 contains the components'-Jo 0i 2 3 1 E*, E63, 6* U 3 contains the components e'*, e4, e, and whl 3 43' et 4' 2 whr U- "a2, * where U contains the components e*, c, E.

CHAPTER VI THE DISCRETE TIME DOMAIN CHEBYSHEV APPROXIMATION PROBLEM OF NETWORK SYNTHESIS 6. 1 Introduction In this chapter we shall consider the application of the results of the previous chapters to the discrete time domain approximation problem of network synthesis. Specifically, we shall consider the following problem: Given a prescribed impulse response, h(t), find the best vector pair (ol**, s*) ines x P such that if the approximating impulse response function is given by n s *t h**(t) = a** e k=1 then l **(ti)lo lh(t)- h**(ti)l, i = 1 2,.. q (6.1) is minimum, at the q equally-spaced discrete values of t, {ti t + (i- 1)At: i = 1,...,q, q > 2n}. Recall that in Section 1. 4 (Chapter I), we reformulated this problem in terms of an approximation problem in Iq -space which we then 1Recall that in the case when the components of s* are not distinct, the function h**(t) has the form given by Eq. 1.3a (Chapter I). 248

249 solved in Chapter IV. We shall now take the results of that chapter, i.e., the vector pair (f**, z*) in 9z x 5 and determine the equivalent vector pair (a**, s*) intYs x J2. The condition, which must be satisfied by the vector pair (a**, s*), or (_f**, z*), so that the approximating function h**(t) represents an impulse response of a physically realizable network is given in Section 6. 3. In Sections 6. 4 and 6. 5 we shall consider the problem of selecting the finite approximation point set and the dimensions of the parameter vectors. Then, in Section 6. 6, we shall give a procedure which outlines the application of these results to the R-L-C network synthesis problem. Finally, we shall present a simple illustrative example of the synthesis procedure. 6. 2 Determination of the Vector Pair (a, s) from the Vector Pair (_, z) In Section 1. 4, we have given the mapping which takes the vector pair (a, s) in 4s x 9Y into the vector pair (_, z) in z x. This mapping is defined by the following equations skt ki Ok = ak e, k = 1,2,...,n (6. 2) and skAt k Zk = e, k= 1,2,...,n (6. 3) where t1 is the initial value of the finite set of discrete equallyspaced values of t, taken at an interval At.

250 In this section, we shall consider the inverse mapping, i. e., the mapping which takes the vector pair (a, z) in Bz x ~ into the vector pair (a, s) inds x 9P. This inverse mapping is not oneto-one as shown by the equations k = [lnlz + j(arg k+ 2m)], m = 0, +1, +2,... (6. 4) = Sktl ( wherek k 1,. However, it should be noted that if only the where k = 1,2,... n. However, it should be noted that if only the principal value (i. e., if m = 0), is considered, then this mapping 2 is one-to-one. There is one special case, specifically, the case when zk is real and negative. Let z0 denote the real negative component of the vector z. Then, when zk = z and m = 0, Eq. 6.4 yields the exponent s = t [nlz01o + j7] (6. 6) where z iz0i =I = Iz e (6.7) This implies that a single negative real component of z yields a single complex exponent, so. Clearly, this sO is not in the set, Note that one may choose some other value of m / 0. However, if m j 0, then, one must consider both the positive and negative values of m, to obtain a complex conjugate pair of the exponents sk.

251 since it yields a single complex exponential approximating function, namely, St so(t-t1) (%+jwo)(t-t1) a0 e = 0 e = e (6.8) where o t [inlzOI], and (6.9) = At [inIz0 =Lt (6.10) 0 =~At We shall now show how to obtain a complex conjugate pair of exponents from a negative real component z0. Let us represent z0 by the sum of two components z0 and z02, i. e., -1! 2 z ('z + Z2) 2 z0) where zo =-Izol Izol ej, and (6.11) 1 0 Z =-lz0 A 1z01 e-j (6.12) Hence, the pair (%,z0) may be represented by the vector pair o/2J' 0 ) (6.13)

252 Applying the mapping defined by Eqs. 6. 4 and 6. 5 yields the vector pair a([a 0]'[:1) (6. 14) where 0S = 0 + jw0 02 = a0 J- 0 and s t1 0 1 0 - 2 -s t 00 02 1 02 2 and where aO and c0 are given by Eqs. 6.9 and 6. 10, respectively. Clearly, the vector pair given by Eq. 6. 14 is incs x J, since the resulting exponential function is real, that is, 01 2 h(t) = 0 e + + e = 0 ea0(t-tl)[eJo(t-tl) + ejo(t-tl)] a (t-tl) =0 e cos wo(t-tl) (6.15)

253 Some reflection reveals that Eq. 6. 15 is simply the real part of Sot Eq. 6.8, namely, Re{aO e } In summary, the vector pair (a, s) e j x Ps of the approximating function n skt h(t;a,s) = ak e (6. 16) k=l or alternately, ~ i sit n skt h(t;a,s) = (a + a2t +. a.t ) e + a esk (6. 16a) -J- k=j+l1 is obtained from a given vector pair (3,z) E x a as follows: 4 (1) For each real non-negative or complex Zk, the components {ak, sk} are obtained from Eqs. 6. 4 and 6. 5, when m = 0. (2) For each real negative Zk, one has to use the equivalent exponential function given by Eq. 6. 15. 6.3 Physical Realizability In the previous section we have determined the vector pair (a**,s*) in s x,7 from the vector pair (**,z*) in z x 5 3 Note that Eq. 6. 16 gives the format of h(t; a,s) when the components of s are distinct, and Eq. 6. 16a gives the format of h(t; a,s) when the components of s are not distinct, namely when s = s2 =...= sj. 4Note that when zk = 0, for some k, then the function eskt represents a unit impulse function.

254 We shall now consider the necessary condition which must be satisfied by the vector pair (a** s*), or (** z*), so that the approximating function5 n s*t h**(t) h(t;a**,s*) = * e (6.17) k=l may be realized as a network function. Recall from Section 1. 2 that the approximating function, selected from the set,a = {h(t;a,s): a ea s, s e }, represents an impulse response function of a linear, passive, lumped, finite network, if all the sk's have negative real parts and if each sk having zero real part is simple. Hence, we shall say that the approximating function h**(t), defined by Eq. 6. 17, is physically realizable if the vector pair (a**,s*) ec x satisfies the following condition: Condition PRs: The vector pair (a, s)e'Js x 92 is said to satisfy Condition PR if the components of the vector s e 9P s satisfy (1) Re{sk} <, k = 1,2,...,n, and (2) when Re{sk} = 0, then sk s sj for all k j Since the approximation problem, considered in this thesis, seeks the parameter vector pair (**,z*) e z x, it is convenient to 5Note again that in the case when the components of s* are not distinct, then h**(t) has the form given by Eq. 6. 16a. The results presented in this section apply for all s e 9,2.

255 translate ConditionPR in termsof the vector pair (_,z). This yields the following: condition: - -,! f,0. -... -..; Condition PRZ: The vector pair (3,) ze Z x is said to satisfy Condition PR if the components of the vector z z e satisfy (1) Izkl < 1, k = 1,2,...,n, and (2) when Izkl = 1, then zk z forallk j. IC -K- f f - 0,.-. satisfiesJCondition.-PR Hence, if the vector pair (1**,z*) e x Z satisfies Condition PRz, z Z then the corresponding approximating function h**(t) is physically realizable. It stands to reason that, by using Definition 3. 1, one can obtain a condition on the vector r* E.which guarantees that h**(t) is physically realizable., However, since such a condition is quite involved (Refs. 4, 9), we shall not examine it here. The assumption of Condition PR immediately bridges the gap between the approximation problem considered in Chapter IV and the time domain approximation problem of network synthesis. It is desirable, however, to know at the start of the approximation problem that Condition PR will be satisfied. Clearly, such a problem inovlves the z solution of the Chebyshev approximation problem, of Chapter IV, with constraints. In other words, since the Condition PR restricts the z vector z to some subset, $PR' of the set, then one must seek the vector pair (j,z) e Z x g, but among those (~,z) e z x pR.

256 However, since Condition PR depends also on the selection of the finite point set Te = {ti i = 1,...,q} with respect to the behavior of the prescribed h(t), we shall not examine the approximation problem with constraints here. Instead, we shall attempt to choose the point set T to guarantee the assumption of' Condition PR. This e z will be discussed in the next section. 6. 4 Selection of the Finite Approximation Point Set, Te Let us now state some of the considerations which dictate the choice of the finite point set T ={ti: i = 1,2,...,q} over which e the approximation is to be performed. Recall that since the elements {ti} of Te are related by t = t + (i-)At, i = 1,2,...q (6.18) where At is the interval between the equally-spaced values of t, then the point set T is fully specified, once (1) the approximation interval e [t1,tq], and (2) the number q have been selected. Let us consider these two problems separately. (1) Approximation interval [t1t ] should be chosen so that the solution of the approximation problem will give a vector pair (B**,z*) which will satisfy Condition PR. For example, assume that the,....'~.. ~;'. -~ -.:.. —.- ~- ~...~

257 ht) ti t t' q q Fig. 9. Example of a prescribed impulse response function h(t).

258 impulse response, h(t), shown in Fig. 9 is to be approximated over a specified interval [tl, tq] After performing the approximation if the vector pair (**,z*) does not satisfy Condition PRZ, then by enlarging the approximation interval to [tl,t'] one might assure that q the new (*, z*) will satisfy Condition PR z (2) The number q should be chosen so that the value of lle(t)lioo = h(t) - h**(t)llo is small7 in between the discrete values of {ti}. This implies that the interval At must be smaller than the smallest interval in which h(t) varies between relative maximums and minimums. 8 However, since the number q is directly related to the amount of numerical work involved in performing the approximation, then it should be as small as possible. In summary, then, the choice of the finite point set T dictates the e value of 11e (t) llI for all t e [t, t], and the physical realizabiIity of the approximating function h**(t). Note that since Fig. 9 depicts a physically realizable impulse reo0 sponse, h(t), i. e., f I h(t) I dt < oc, one expects to obtain a (**, z*) which satisifes Condition PRz. 7Recall that ideally one would like to approximate h(t) over the continuous interval [tl,tq]. 8Note that the value of At can be obtained from the Sampling Theorem, io e., At = 12, where W is the highest frequency of the waveform h(t).

259 6. 5 Choice of the Dimensions of the Parameter Spaces In choosing the dimension n, of parameter space s9z x a one must consider the effect of n on the complexity of the resulting network, the tolerable error, Ie ** lo, and the amount of computational work. With regard to the complexity of the network, it is seen that since the approximating function is a linear combination of n exponential functions, then the resulting network will have between 2n and 3n elements. The relation between n and the value of le**l100 is more complicated. All that one can say is that the number n is inversely proportional to the value of 11 **l. Hence, when the value of n is increased, the value of 11l**ll1 is decreased. However, this holds only up to the point when n = 2 since then, lle** l = 0. 2 00 Another difficult relationship to define is the one between the amount of computational work and the number n. However, in general the larger the number n, the more computational work will be involved in solving the approximation problem. Therefore, the selection of the number n depends on a balance between these three factors. 6. 6 Synthesis Procedure Now that the preliminary ideas have been discussed, let us consider the computational procedure to be followed in approximating a prescribed impulse response, h(t), which is given in analytic or graphical form:

260 (1) Select the finite point set Te ={ti =t + (i-l)At: i = 1,2,...,q} according to Section 6. 4. (2) Knowing the finite points set Te, form the real vector h in Uq from the ordered set of values {h(ti)} of h(t). (3) Choose the number n according to Section 6. 5. (At this point, it is helpful to write out the relation h = [Z(z)] + e, using the value of h determined in Step 2. ) (4) Calculate the parameter vectors (**,z*) in x using the procedure of Section 5. 3. 1. (5) Does (**,z*) e z x 5 satisfy Condition PR? If yes, go to Step 6. If no, repeat from Step 1 by using a new finite point set T e if possible. (6) Determine the vector pair (a**,s*) using Eqs. 6. 4 and 6. 5 (and/or Eq. 6. 15) of Section 6. 2. (7) Form the approximating function h**(t) using Eq. 6. 16 (or Eq. 6. 16a). (8) In the prescribed interval [tl,tq], determine the over-all Chebyshev error 1e (t)1 - sup I e(t) I, where e (t) = h(t) - h**(t). t, [t1,tq] If this 11 e (t) 10 is not tolerable, repeat from Step 1 using a greater number q and/or n.

261 (9) Determine the equivalent network function by taking the Laplace transform of h**(t), namely H**(s) = L[h**(t)]. (10) Realize a suitable network having the network function H**(s) as its driving point or transfer function depending on the desired performance of the network. 6. 6. 1 A Simple Example of the Synthesis Procedure. The following example illustrates the application of the above procedure. Example 6. 1: Let h(t) be given by h(t) = l+t) for t > O = 0 for t < 0 (6.19) Synthesize a network whose output-to-input voltage impulse response function, given by Eq. 6. 16 (or Eq. 6. 16a), approximates h(t) at a finite discrete value of t, in the Chebyshev sense. Solution: (1) The function h(t) is plotted in Fig. 10. The finite point set T is selected to be e T = {0, 0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0} where At = 0. 5, and q = 9.

262 1.0 h(t).8.6.4.2 1 2 3 4 _,.I. I I I 1 2 3 4 5 6 7 8 9 Fig. 10. Impulse response function, h(t) = (l+t)

263 (2) 1. 0000 0. 4450 0. 2-500 0. 1600 0. 1110 0. 0817 0. 0625 0. 0494 0. 0400 (3) Select n = 2. Then the relation h = [Z(z)] _ + e yields Eq. 5. 14 of Section 5. 4. (4) The solution of this problem is presented in Example 5. 2 of Section 5. 4. It yields 0. 31476 ** = 0. 68240 0.76552 z* = 0. 30317 (5) The vector pair (/**,z*) satisfies Condition PR (6) When At = 0. 5, and m = 0, Eqs. 6.4 and 6. 5 yield s1 = 2 in (0.76552) = -0.53444, s2 = 2 kn(0.30317) = -2.38692, and a1** = 0.31476 a2*= 0. 68240

264 Hence, / 0. 31476 ]0. 53444 (a*s,*) = I0.5a 44 0. 68240 -2.38692 / (7) The approximating function, given by Eq. 6. 16, becomes -0. 534t -2.387t h**(t) = 0.31476 e + 0. 6824 e (8) The plot of e(t) = h(t) - h**(t) is shown in Fig. 11. It is observed that sup i(t)ll = t [0,4] Ie(t)l= 0.0118 > l**l = 0.00284 Furthermore, it is seen that if one would perform another approximation over the interval [0, 4] when At = 0.25, then the resulting le (t)loO would lie in the interval 0. 00284 < 11 (t)ll < 0. 0118 and the value of 11 e (t)11 will be closer to the value of II ** I. However, we shall consider this error function, e (t), to be satisfactory and proceed to the next step.

265 010 2 II e **1I * -005 0 0 t ii ~ ~T~1.0 2. 0 3.0 4.70 -. 005 -.010 Fig. 11. Approximation error function, c(t) = h(t)- h**(t).

266 (9) The Laplace transform of h**(t) yields 0.31476 0. 6824 H**(s) s+ 0. 534 s + 2. 387 0. 9972 (s + 1. 1186) (s + 0.534)(s + 2387) (10) A network realizing H**(s) as an input-to-output voltage ratio, i.e., 1 (s) = H**(s), is shown in Fig. 12. C2 -I ii 2 R1 = 1642 C2 1.393028fd R = 2 C3 1.0028fd R3 = 11. 538 Q Fig. 12. Network realizing h**(t) as an input-to-output voltage ratio.

CHAPTER VII CONCLUSIONS AND AREAS OF FUTURE WORK The discrete time domain approximation-problem of network synthesis was originally posed in Chapter I. In particular, the problem of determining the coefficients and exponents of the exponential functions when a prescribed impulse response function, h(t), is approximated by a linear combination of exponential functions so that the resulting error is minimum in the Chebyshev sense., at a finite number of equally-spaced discrete values of t. This led to the Chebyshev approximation problem considered in Chapter IV, where both the best orientation of the approximating subspace and the approximating vector in it were determined for any prescribed vector in a finite dimensional vector space. In this chapter we shall summarize and evaluate these results and point out the various extensions which can be carried out in the future. 7. 1 Discussion of Results The primary contributions to the solution of the problem considered in this thesis are: (1) The formulation of the time domain approximation problem "of network synthesis in the language of the theory approximation in 267

268 linear spaces. In particular, we have formulated the discrete approximation problem in terms of an approximation problem in the finite dimensional vector space. Such a formulation depicts the direct relation between the pole position of the network function (i. e., the exponents of the exponential function) and the orientation of the approximating subspace. (2) The formal extension of "Prony's Original Method" to the solution of discrete exponential approximation problems, which is presented in Chapter III. Such an extension gives an alternate approach to the solution of exponential approximation problems. It illustrates the limitations of the previous works by Ruston (Ref. 20) and by Yengst (Ref. 26), which use similar approaches. (3) The solution of the Chebyshev approximation problem in the finite dimensional vector space when the approximating subspace is a function of a collection of n-parameters. The main results of the theory, presented in Chapter IV, are: (1) the existence theorem; (2) the bounds within which the minimal value of 11 e 1 must lie; (3) the property which states that the best Chebyshev parameters define the best Chebyshev approximation in some (2n+1)-dimensional reference subspace; and (4) the conjecture which states that the minimal value of \11, occurs when at least (2n+l)-components of i are equal in absolute value to lel\c if there exists no other

269 approximation which yields a smaller value of l e lco when 2n-components of e are equal in absolute value. If, however, there exists an approximation which yields a smaller value of 11 Ili when 2ncomponents of e are equal in absolute value, then it is conjectured that such an approximation represents the best Chebyshev approximation. (4) The method of descent used to solve the Chebyshev approximation problem. It is based on "Prony's Extended Method, " and is presented in Chapter V, where a computational algorithm is given. This method systematically decreases the value of 11e1 until (2n+l)-components of the error vector, c, are equal in absolute value to 11 Ell At this point, the error vector is tested to determine whether or not there exists another error vector, i, which gives a lower value of \li Fl, when only 2n-components of e are equal in absolute value. The computations required for this test lead to the next step of descent, if it is required. (5) The approximating function is such that its Laplace transform can be expressed in rational function form. This rational function is not restricted to a class of functions containing only first-order poles, since for each repeated exponent, sk, the approxiskt skt 1 mating function possesses terms of the form {e, t e,..,t e where j denotes the order of the repeated exponent, sk. This result

270 follows directly from "Prony's Extended Method," presented in Chapter III. If each exponent, of the approximating function, has a negative or zero real part, then the rational function is suitable for realization as a driving point or transfer function. 7. 2 Limitation of the Results The significant limitations of the results presented in this thesis are concentrated in their application to the time domain approximation problem of network synthesis. The approximation theory presented in Chapter IV is limited only in the fact that the resulting approximation might not be unique. However, the significance of this limitation is diminished by the fact that in network synthesis the problem of realization is usually not unique either. The limitations encountered in applying the approximation theory developed here to the time domain approximation problem of network synthesis are: (1) The form of the matrix [Z(z)] dictates that the prescribed impulse response function must be sampled at equally-spaced intervals of t. (2) There is no control of the behavior of the error function, (t) = h(t)- h**(t), in between the equally-spaced discrete values of t.

271 (3) The solution of the approximation problem does not necessarily yield a physically realizable impulse response function. Recall that the requirement that the approximating impulse response function be physically realizable was considered only after the approximating function had been determined. 7.3 Future Areas of Study The following areas of further study were revealed during the course of the investigation reported in this thesis: (1) The problem of proving Conjectures 4. 1 and 4. 2. This is the problem of showing that Conjectures 4. 1 and 4. 2 give the necessary and sufficient conditions for the best Chebyshev approximation in each (2n+l)-dimensional reference subspace. It involves the study of the behavior of the nonlinear equations which relate the (2n+l)components of the error vector in each (2n+l)-dimensional reference subspace. (2) The problem of determining the necessary and sufficient conditions for the uniqueness of the solution of the Chebyshev approximation problem, considered in Chapter IV. This investigation should yield a certain class of prescribed vectors which will always result in a unique Chebyshev approximation. (3) The problem of determining the necessary and sufficient conditions for the approximating function to be physically realizable.

272 This is the approximation problem of Chapter IV with constraints, where the constraints are imposed on the parameter vector z, i. e., z must lie in the parameter space YPR' defined in Section 6. 3. (4) The problem of selecting the finite approximation point set, T, and the dimension, n, of the parameter space, so that the value e) of l c (t)lI II h(t) - h**(t)llc is small in between the equallyspaced discrete values of t. (5) The problem of approximating a prescribed impulse response, h(t), over a continuous bounded interval of t, [a,b], so that the resulting error Ue (t)U is minimum. Some thought should be given as to whether the approach of using the orthogonal complement subspace to the approximating subspace may be used to solve this problem. A similar approach was used by McDonough (Ref. 11) in determining the best least-square approximation over the continuous interval [0,oo). It should be mentioned that in the case of the Chebyshev approximation the resulting error function should, in general, alternate (2n+1) times in the approximation interval [a,b]. (6) The problem of finding an efficient computational method should be examined so that this method can be directly adapted for digital computer use.

APPENDIX A STIEFEL'S ALGORITHM A. 1 Theoretical Analysis In Section 2. 3. 1. 2 of Chapter II, we mentioned the "method of ascent" used by Stiefel (Ref. 22) to solve the Chebyshev approximation problem. We shall now present this method in considerable detail, giving the iterative computational method and an illustrative example. Stiefel sought the best Chebyshev approximate solution, {xj*: j = 1, 2,..., n} of the overdetermined system of linear equations given by n fi = a.. x. i=12,., q (A. 1) j=l U where the sets {fi} and {aij} are initially prescribed, and where q > n. Instead of presenting Stiefel's method of solving Eq. A. 1 verbatim, we shall present his method in the setting of this thesis, i. e., in terms of operations in a finite dimensional real vector space Eq. Recall, from Section 2. 3. 1. 2, that the problem of solving the overdetermined system of equations of Eq. A. 1 may be stated as follows: Given a vector f in Eq and a qxn matrix [A], of rank n (q > n); find the parameter vector x* in En such that if f* = [A] x*, then 273

274 ll*ll - llf-f*ll < llf- [A] xll (A.2) for all x x x* in En. Hence, we shall be concerned with the vector relation f = [A]x+e, in Eq (A.3) where the initially prescribed vector f and the qxn matrix [A], of rank n (n < q), are given by f f =, in Eq, and (A. 4) f a11 a~ "in all aln a21 a2n [A] = 2 n (A. 5) a a aql. qn and where the unknown vectors e and x are defined by = = 2, in Eq and (A. 6) 6 q

275 x1 x =., in E (A. 7) x Furthermore, since the method of solving Eq. A. 2, proposed by Stiefel, is based on Corollary 2. 1 above, we shall be also concerned with the vector relation f() = [A()] x + in En (A. 8) where the (n+l)-dimensional vectors f() [A(v)] x, and e( in En+l are the respective projections of the q-dimensional vectors f, [A] x, and e in Eq onto the v-th reference subspace, En+ of E, according to Definition 2. 1. Recall that Corollary 2. 1 states that the problem of minimizing the value of 11e I, i.e., 1I e* 11 = min Ie (x) 11 (A. 9) OC 00 x in E where e is defined by Eq. A. 3, may be replaced by determining the largest value of 11e (xv) )I from the set {l (v ): v= 1 2. (nql)}, namely,

276 11le*l = max {ll1 ()x*)11_} (A. 10) l<v<( q) - - n+1 where the vector E (x*) is given by Eq. A. 8 when x = x* and where X'* defines the best approximating vector, f*V) - [A(V)] x* to f(V) in E n+ 1 E l, and is determined by using Theorem 2.3. Hence, the method of solution of Eq. A. 2 proposed by Stiefel is based on the method defined by Eq. A. 10. Basically Stiefel's algorithm is as follows: We begin by arbitrarily picking an (n+l)-dimensional reference subspace, say, Ek, where k e {v=1,2,..., (nq1)}. This yields the relation f(k) = [A(k)] x + (k), in Ek1 (A. 1) Then, by applying Theorem 2.3 to Eq. A. 11, the vector xk* in E which minimizes e(k) oII and the vector (k) (x) in En+ are determined. Knowing the value of x*, the value of the vector e(xk*) in Eq is determined from Eq. A.3, whenx = k*. At this point we compare the I -norms of the two error vectors e (x)k* and E(x), and see if they satisfy either II E(x*)0II = (xk*) l (A. 12) or

277 (k) eC(Xk*)llcc > ll( (xk*)ll (A. 13) If Eq. A. 12 is satisfied, then the vector k* = x*, where x* is the vector x which prescribes the solution of the Chebyshev approximation problem defined by Eq. A. 2. If, on the other hand, Eq. A. 13 is satisfied, then a new estimate of x* must be sought. Since we are seeking an estimate of x*, which will increase the value of le(k)(xk*) l so that eventually Eq. A. 12 will be satisfied, then the new estimate of x*, say, xm, must be chosen so that the new value of lle(x *) ll, where e(xm*) is given by Eq. A. 3 when x = x *, will satisfy [[ x E (_Xk* [ > HE(k),I IlE(k*)loc > IIe(x*) > lE (xk *)l or alternately, Il > _Xk*) (i) *(k) -k (X )E (xm)c > ime (xk)I) (A. 14) Hence, we have the problem of finding a new (n+l)-dimensional reference subspace E which will yield an x * so that Eq. A. 14 will be m -m satisfied. The method of solution of this problem represents the contribution of Stiefel, and is summarized in his Exchange Theorem (Ref. 22). Now, before considering Stiefel's Exchange Theorem, let us make the following observation: When the initial estimate of x*, i. e., Xk* yields an error vector e(xk*) whose t -norm satisfies Eq. A. 13, there ""~ ~K O~Cc

278 exists at least one component of e(k*) in Eq whose absolute value equals to lle(xk*)ll. Let us denote this component by eb(k*), where b(xk*) satisfies leb(xk*)I = le(k*)l (A. 15) and where the component eb is not one of the components of the vector e in E which formed the vector e(k) in Ek + Then, the subspace.....' which contains the subspace En+ plus the b-th component of the vector which in e bsae E+2 in E is an (n+2)-dimensional subspace, E n+2 This can be best illustrated as follows: Let the vector relation in Ekn be given by 2fall 21 a2n l2(X) + (A. 16) a.. ax 2 _x n+1 an+1l an+l, n n+l) where f in E and the (n+l)x nmatrix [A] are prescribed. Applying Theorem 2.3 to Eq. A. 16 yields the values of the vectors x* and e(x*) where the components of the vector e(x*) are given by To simplify the notation, we shall drop the superscript and subscript k in Eqs. A. 16 through A. 20 and Theorem A. 1, since there is no danger of ambiguity.

279 Ei(x*) = p sgn i, i=1,2,..., n+l (A.17) Let us assume that when substituting the value of x* into Eq. A. 3, the component en+ 2(x*) of the vector (x*) e Eq satisfies2 IEn+(x*) > Ipl (A. 18) We now form the following vector relation which defines the (n+2)nit2 dimensional subspace E, namely, f l11 1,n l 1 Psgnl1 f2 a21. a2 p sgn E2 =: - ~:: (A. 19) ~ a a p sgn E fn+1 al,+1 an+1, X n n+1, n+2 a n+2, a En+2 n+2 where the vectors f, [A] x and e in E are all known, and where the value of I n+2I > Ipl. We now have the problem of finding the (n+l)E n+1 n+2 dimensional reference subspace E of E so that the parameter vector x, which defines the best Chebyshev approximating vector in E + according to Theorem 2. 3, yields an error vector e(Xm*) e En m mwhose ice(x *)ll satisfies Note that we are also assuming that n+(x*) I = I e(x*)I, where E(x*) e E, i. e., that Eq. A. 15 holds when b = n+2.

280 I n+21 > _IIE(Xm*)1 > Ip (A.20) The solution to this problem is given by Stiefel in the following theorem: Theorem A. 1: (Exchange Theorem). Let the vector [A] x,where [A] is an (n+2)x n matrix, of rank n, and x is in En, represent the n+2 approximation vector to the prescribed vector f in E such that the resulting approximation error vector e has the form p sgn e 1 p sgn e2 e f-[A]x = [ (A. 21) p sgn Enl en+2 where en+21 > Ipl (A. 22) Then, there exists an n-dimensional vector y x in E, such that if [A] y is another approximation vector of f in En, then the approximation error vector 71 has the form

281 sgn'71 sgn 7j_ 1 f - [A] = (A. 23) 6 sgn 7rj+ 6 sgn 7n+2 where j e {1, 2,..., n+1}, and where 6 satisfies Ip < 161 < Ien+21 (A.24) Proof: Let us begin by considering Eq. A. 19. Assume that the two independent vectors in En+2, which span the orthogonal complement subspace of the n-dimensional subspace Cn(A), are given by3 1 JL1 x =_. and j( = (A. 25) kn+ n+1 0 3It should be noted that the vector is also orthogonal to the projection of Cn(A) on the (n+l)-dimensional subspace En+ formed by the first (n+l) components of Eq. A. 19.

282 Then, applying the orthogonality conditions ([A]x, _) = 0, and ([A] x,,) = 0, to Eq. A. 19, we obtain the following two equations: n+1 (f - E,) = X1 (f-Ei)= (A.26) n+1 (f - E,) = Z ~i(fi-ei) + (fn+2+En+2) (A.27) i=1 Multiplying Eq. A. 26 by - j and Eq. A. 27 by X. and adding the resulting equations, yields n+ 1 Z (iA. j- Ai) (fi - i)+ X(f - E ) 0 (A. 28) ti ~ j ij. i ( fjn+2 n+2 tij This equation may be written as n+1 n+1 /i Aj /i Aj Let us denote the coefficients of ei s and fi's by vi, i. e., let V.i (A.30) Then, Eq. A. 29 becomes

283 n+1 n+1 3 ir.E +~2 n~ = ~f.+ (Af31) i=1 i= (A.31) ilj inj Applying the lemma of de la Vallee Poussin (i. e,, Lemma 2.1, Chapter II), to A. 31, we find that the minimal value of the expression max { eil} i=1,..., n+2 igj is given by n+1 i fi + n+2 = n+1 (A. 32) 5 lv.1 +1 i=l itj Observe that Eq. A. 32 gives an alternate representation to Eq. A. 31, namely, n+1 n+1 vf i n+f2 6 vl +1 (A. 33) i=l I i =1A 31 ifj.Li/ij Equating Eq. A. 31 to Eq. A. 33, yields

284 n+I n+l iei. + e 6 Ivil + 1 (A. 34) 17 —i 17 itj,itgj or;i \\ i x^) - ni iil+ i=n l. i + 1n+ X.6. i I iP1 (tI) Xici+En~2jI= i nx (A. 35) Recall from A. 21 and Lemma 2. 1 that X. = I Xlp, for i=1,..., n+ (A. 36) then, by taking the absolute value of both sides of Eq. A. 35, one obtains l,.tn+l ("+ E3;. en+2 n+l ILLi _. iIj. Ix I 161... I+1 i~l \\ ^/ ^ P i=l i X. 1 J (A. 37) Since n+2 I > Ipl by definition (i. e., Eq. A. 22), we find that if the relation lp < 161 (A. 38) is to be guaranteed, then the sign of e n must equal the sign of

285 that is, (a) if en+2>, then - > 0, or (A, 39) (b) if n+2 < 0 then (X) 0 (A. 40) To show that n+21 > 161, let us consider Eq. A. 37 under either the condition of(A. 39), or (A. 40), depending on the sign of en+2 This yields len+21 - 161 = [11 - ipl]. I- x. (A. 41) i=1 1 J ifj Since the right hand side of Eq. A. 41 is greater than zero, then en+2 > 16 (A.42) and the theorem is proved. It is noted that this method does not necessarily increase the deviation 156 at the fastest possible rate since Theorem A. 1 does not guarantee that 1651> 171j, j= 1,2,..., n+l (A.43) -(J

286 However, by considering all possible n+l values of 16. 1, given by Eq. A. 32 for each j, that is, E_ | l~- 1|nI I n+2 i^j (A. 44) and choosing the j which maximizes 16j, will result in the fastest possible increase in 1 1 for each replacement step. Furthermore, the resulting error vector 1, defined in Eq. A. 23, becomes the best Chebyn+2 shev error vector 1j* in E, that is, 11X*11 = 161 Hence, the resulting vector [A] y* is the best Chebyshev approximation vector of f in En+ Now before presenting Stiefel's algorithm in detail, it should be mentioned that there are other algorithms which converge faster than Stiefel's point-by-point exchange algorithm. One of these is the method proposed by Remez (Ref. 17) which exchanges all the (n+l) points at each step of the algorithm, i. e., it selects a completely new (n+l)dimensional reference subspace E. We shall not consider this - m method here and the reader interested in this method is referred to the treatment of this subject by Rice (Ref. 19, pp. 176-180).

287 Let us now present the step-by-step procedure proposed by Stiefel for the solution of the Chebyshev approximation problem of Eq. A.2. A. 2 Stiefel's Iterative Procedure (1) Write out Eq. A. 3, substituting for f and [A] their respective prescribed values. (2) Select any (n+l)-dimensional subspace out of the set of Enl v= 1,2, (n+), in E Let us denote it by Enl, where k e v= 2,... (2n+') Write Eq. A. 8 in the form (k) f(k) = [A(k)] x+e(k), in Ekn+ (A.45) where f (I) is an (n+l)-dimensional vector representing the projection of the prescribed q-dimensional vector f onto Ek+ and similarly, [A(k)] is an (n+l)xn matrix formed by selecting the appropriate rows from the prescribed qxn matrix [A]. (3) Find the (n+l)-dimensional vector X(k) in En+ which is orthogonal to the column space of [A(k)] C (A(k)) from [A(k)] (k) 0 (A. 46) where

288 (k) x,(k) - (k) n (k) Ck) Note that if the (n+l)xn matrix [A(k)] is of rank n, then the vector X(k) can be determined directly. (4) From Eq. A. 45 and from the fact that ([A(k)] x, (k)) = 0 for all x in C(A(k)), form the equation4 ((k) X(k)) = (f(k) (k)) (A.47) f (ki(k) = c (A. 48) i= 1 (k) (k) where f (k) and k) are given in Steps 2 and 3, respectively. (k) (5) From Eq. A. 48, determine the vector (k), whose C -norm is minimal. This vector (denoted by E*(k)) is given by Lemma 2. 1 as follows:,(k) sgn e 1,4 s(k) g * (k) 4kk n (k) This equation implies that the orthogonal projection of f(k) and e(k) on x(k) are equal.

289 where C Pk n 1 + z li(k)i i=1 l and,(k) s (k) sgn ei*( = sgn.(,,, n+ (6) Determine Xk*, from any n rows of the vector relation [A(k)] xk* = f(k) *(k) (k) where e*( takes the value found in Step 5. (7) Solve for e in Eq from Eq. A. 3 whenx =k*, where k* was found in Step 6, that is, (x*) = f - [A] xk (8) (a) If lle(Xk*)llo = IPkl thenxk, the best Chebyshev solution vector of Eq. A. 3. (b) If le(Xk*)loc > Pk[ go to Step 9. (9) Let the b-th component of e(Xk*), determined in Step 7, be the component whose absolute value equals to lle(xk*) II, i. e., Ieb(Xk*)l = lle(Xk*)11c

290 n+1) Then, form a new n+1-dimensional subspace (denoted by E ) by using Theorem A. 1. This will be done in Steps 10 through 12. (10) Calculate the (n+2)-dimensional vector j defined by -L2 An+1 1 from the equation [A(k)] e —-0 trix Ab is the b-th row of the qxn matrix [A] equivalent to the component Eb. (11) Form a set o (n+) values of () val the ratios of the components of (k) the vector L to those of the vector () found in Step 2, that is, 5Note that this equation will yield a vector, which is orthogonal to the (n+1)-dimensional subspace, spanned by Cn(A) and the vector X. Although this simplifies the calculation of A, it suffices to find a vector J which simply is not in the (n+1)-dimensional subspace.

291 (k) | where i= 1,2,..., n+l (A. 49) n+1 (12) Determine the coordinate j, of the subspace E, to be replaced by the coordinate b, as follows: (i) If eb > 0, then j is the subscript i which corresponds to the smallest element of the set k) defined by Eq. A. 49, that is, I ^ min i k) 1_< i< n+1 i(k) (ii) If Eb < 0, then j is the subscript i which corresponds to the largest element of the set i (k)defined by Eq. A. 49, that is, max (k) 1< i<n+l (k) (13) Form the new vector equation f(m) [A(m)] x+(m), in Em where the (n+l) vector, f(m), and the (n+l)xn matrix, [A(m)], correspond to f(k) and [A(k)], respectively, when the j-th row has been replaced by the b-th row of the vector relation given in Step 1.

292 (14) Determine (m) and x * by repeating the procedure starting with Step 3. (15) This procedure is repeated until the relation in Step 8 is satisfied. A. 3 Illustrative Example Given: 4 1 2 2 3 4 = and [A] = 3 -3 2 1 Find the vector x* in E, such that iff* - [A] x*, then, 11*11 ^ llf-f*ll < fll- [A]x 11 for all x 4 x* in E2 Note: q=4, n=2. (1) Equation A. 2 becomes 4 1 2 x e = 4 x2] + 2 (A. 50) 7 23 -3 2 1 4 (2) Since n= 2, then Ek = Ek Hence, select any 3 component of Eq. A. 50 to represent the vector relation A. 45. Let k= 1, then

293 Eq. A. 45 becomes ^f() = lA(1)]x + e(1) in E3 where 4 E1 f(1) 2 (1) 1- 2 [A]3 4 3 2 3 ^2 (3) Equation A. 46 becomes 1 3 2" (1) -~ iL ~ Solving for ki() from this equation yields -0. 5 (1); -0. 5 1 (4) Equation A. 47 becomes

294,c (_.e(1) X ( 1)) (f(l).(1)) =4(-0.5) + 2(-0. 5) + 7(1) = 4 (5) The value of c 4 =~=2 3 2 Ixi(')l Therefore, sgn k -2 () = 1 P sgnk2 -2 sgnX 2 (6) Determine x from [A(1)] x* = f(l) *(1) 1 2 x4 -2 6 34 1X2 = 2 -2 = 4 2 3 x 7 24 5 Solving this equation yields *= -8

295 (7) Substituting x1 into Eq. A. 50 and solving for e yields, 1 4 12 -8 -2 = C2 = 2 3 4 L7 63 )7 2 3 2 4-3 2 1 6 (8) Since l(xl*) II = 6 > Ipl- = 2, go to Step 9 of procedure. (9) From Step 7 of procedure, b=4 since e4(x1*)1 = 6 = Ile(xl*)ll (10) Determine the vector! by solving the following equation: [A(1] (1)T 1 3 2 2 l.1 /- I 2 0 -.-. —- M0. 5 -0. 5 10 o Ab 0 Hence, 7/3 -5/3 1/3 (11) Form the set defined by Eq. A. 49

296 l i,!-14 110 1i) 3' 3 3 (12) Since 4 = 6 > 0, found in Step 7, then j= 1 because -14 /~i = m )(min ) of Step ll (13) Form f(2) [A x in E f(2) = [A(2)] x +(2) in E2 where f^ - 7; _E 03 f(2)= (2)= E1 [A(2)] 2 3; and x= (2 1 F2 (14) From 3 2 2 x (2.) [A(2)]T t(2) 12(2)

297 one obtains -4 ~(2)= []5 (E(2) A(2)) = (f(2) x(2) = 24 24 24 P2 3 = 1 =2. 4 (2)I i=l Therefore, -2.4 () = 2.4, and 5.2 Substituting 2 for x in Eq. 50, yields, Substituting x^2* forx in Eq. A. 50, yields, -0.8 -2.4 ((X2*)= 2.4 (A. 51) 2.4

298 Since II(x2*) I = 2 4 = p, thenx2* =x* i. e., the best Chebyshev solution vector. To summarize, the vector x* of Eq. A. 50 is -5. 2 x*= 5. Hence, 4.8 4.4 * [A]x* = 4.6 and -5.4 II* [I = 2.4, where e* is given by Eq. A. 51. 0c

APPENDIX B PROOF OF THE RELATIONS OF LEMMA 4. 1 GIVEN BY EQ. 4.9 THROUGH EQ. 4.14 In this section we shall prove the following relations: (1) If z # 0 and if z -o1 < 2 Zoloo then 1(l*(Zo),Z)llc - ll( *(z)ollc < c(zo) l - Zolll (B. 1) where -1 -1 1 c l(Zo) =2 11 f oo 1 zoi (qj) (B. 2) ='-o 2 K II-I cc~(B.2) and zll-)lloc-{l Z))llo < c 2(Z) ll-Z olll (B.3) I- 2 -0 where q-1 z C P~1: (B. 4) c2(Zo)= 2(n+1) 11flo 11 (1 (B.4) j=l (2) If z = 0 and if lIzll1 < 1, then ll((Zo)z)lo - lle(*(o)Zo)loo < 2 1fl (B. 5) and ll(f*(_z),_z)lio - ll~(*(_z),_Zo)ll0 j < 2f.lloc (B. 6) 299

300 First, we note that since I(1*(),z) iZ I l- [Z(z)].*(z)l)o > II [Z(z)] *(z)11O - IIfll, (B.7) and II(P-*(Z)z)~lloc O llllI00 (B. 8) then _] )l <iq Z mak*(x)zk < 211fll (B.9) oo l<<_q k=l Pk* (-i) Z k 00 Furthermore, we note that the binomial expansion yields [ - zm] = [z (z_ - - Zm]) 0 0 0 m - ( 3 ) zm [z ]J (B. 10) (1) Using the relation given by Eqs. B. 9 and B. 10, let us now obtain the relations given by Eqs. B. 1 through B. 4 when z 0 O as follows

301 I ll( (Zo),z)lo - le (o0)lo < ll (*(Zo), z) - e(*(Zo),Zo) l I [Z(zo) - Z()] P*(Zo)!lo max Q r -'] = i 1 k*(Zo) Z z Ik f kl (z_)k J= l J k| kZ J i-i ~-1 n x max l(_)max i ( 1 max 1 - I kz llz - j Z z z _J=1J -o~=kk=l ~k ~ k - z1f I o < 7 zk- Izz 11j (B. 12) k~l ^ \ " k ~l k thenEq. B. r1 yields lle(*(z ),z)ll- -{le(I*(lz )o l 1<k lfn i (o k j 1o1" UZ-z o (B. 13)

302 Let us now note that i-l Z lZ l =,when i = 1 j=1 (B. 14) -1 = II z o11 lo lz-_z i11, when i =2 = ~~q-1 1|~. ~ (B. 16) (q j) l 1zoLj llz-_ ol, when i =q j1 (B. 16) Furthermore, if liz - zI < 1 Zo1o, then Eq. B. 16 becomes q_ -i 1 9(q;1)!z ll1oj Ilz-z II <!_%l -1Z ( q;1 0 z-_Z111 (B.17) j=l j= ) (lllollZ Hence, from Eqs. B. 13 through B. 17, we obtain Eq. B. 1, namely, lle *(_z ), Z)|l - Ile(*(_Zo),zo)l IO < c l(zo) 11z- 1 (B. 18) where Cl(Zo) = 2Ifl Illooll o j=1l under the constraint liz - Zol <- II zo1, 1 0. Clearly this constraint contains the assumed constraint, i.e., Iiz - Zol < 1 zo11 - -ol- 2 I-o' To obtain the relation given by Eq. B. 3 we simply interchange the variables z and z in Eqs. B. 11 through B. 18. This yields ll_(~*(z), z)ll - 1le(*(Z) Zo)11o < C1(_) lZ -Zolli (B. 19)

303 where q-1 c(_z) 211fl I zLi C (q i) (B.20) under the constraint 1z - Zol11 ~< IZIo. Clearly, if liz Zoll < Izilo when lllzooo # 0; and since I -z oloc <- llZ -zol then o <!1z 11 - llZ - Z 11 lSl lI or in other words 11{ z 1 0. Since z / 0 and z # 0 then from the constraint lzloo > l ol z - ll 11ol - 111z, and from the relation li1 nll z 11, we obtain lzII ~ > 11z l1 - nl z ll or alternately, zlll < (n+l) 1zoll 2(B.21) Substituting Eq. B. 21 into B. 20 yields q-1 c2() = 2(n+1)1lf llZ (- ) c>(z) (B.22) 2 (_Zo) =1_o ll Hence Eq. B. 19 yields Eq. B. 3, namely, I le(,*(z),z)ll - lle(f*()Zo)ll <I c2( )z Zoll (B.23)

304 under the constraint liz - z 1 < izlioo To show that this constraint contains the assumed constraint liz - zol 1 < 2 Izollo, we note that 11 ZoII -llz 1 < Il z - zll = I= z - I < I z - Zoll < lzll that is llZloo > 2 llzol lo (2) When z = 0 Eq. (B. 9) becomes ll[z(o)]P*'(0)l | = lz 1((0) 21l (B.24) Hence, when z = 0, then n I \\e (E(Z) z)110 -l el ((zo), zo) llj I max kI kI) II[Z(z) - Z(O)],(o)IIo - l < ~<_q- 1 () I <(|kil~(1 (<i<q-l ) max i < 211fllx <i<_q-1 -Iz1ll K 2 11fI lzIlI' if IIrlll1 <I (B.25)

= IN e 02 == 1 IN IA IA IA I OF ~_o____ CD, ^ 8 e18 II IN - ___ [ -* -'D IA IA - -N -/5 IN-IN 3 1. 88 _I NN o 8 ~ 9 IA ^I',IN' 8 wC I IIA'o IA _ II'-a.' o NZ

APPENDIX C PRONY'S EXTENDED METHOD IN AN m-DIMENSIONAL SUBSPACE, m > n+l In this section we shall derive the relation given by Eq. 4. 58, namely [A(W)(r)]Tf(W) [A(w)()]T (w) w = 1,.(C.1) where f(W) and (w) represent the projections of the vectors f and E in U onto U2n+, respectively, and where [A()(r)] is a (2n+l) x (n+l) matrix as a function of r e <. Since one can obtain an analogous relation in any m-dimensional reference subspace of Uq, where m > n+l, we shall formulate "Prony's Extended Method" in a general m-dimensional reference subspace Uk which is defined by Definition 2. 1, when m > n+1. In the m-dimensional k-th reference subspace Uk, where k e { 1 = I,..., ()}, the m-dimensional projections of the vectors f, [Z(z)], and in Uq are given by f(k) =[Tf (C.2) Note that one can form (m) distinct m-dimensional reference subspaces from U1q. 306

307 [Z(zk)]( = [Ik]T[Z)] e Uk, and (C.3) (k) rT mU (C.4) r(")=[a^r eu (C.4) where [1k] is a q x m matrix defined by Eq. 2. 25. The relation between these vectors in Uk is given by f(k) (k))] [z(k) (C. 5) Note that [Z(k(z)] is an (m x n)-matrix, where m > n+l. To formulate the relation of Eq. C. 5 in terms of "Prony's Extended Method" we must define a matrix which is a function of2 r e? and whose column space is the orthogonal complement of the column space of the m x n matrix [z(k)()] in Uk. Observe that since the matrix [Z(k)(z)] is of rank n then its column space, Cn(Z(k), defines an m n-dimensional subspace in Uk. Then, the orthogonal complement subspace in Uk must be an (m-n)-dimensional subspace. This (m-n)-dimensional subspace can be represented by the column space of an m x (m-n) matrix which will be denoted by [A(k)(r)]. We shall now show that the matrix [A (k)] can be obtained directly from the matrix [R(r)] by appropriate column and row operations. 2Note that when the vector r is in A!, the relationship between the q x n matrix [Z(z)] and the q x (q-n) matrix [R(r)], given by Lemma 3. 1, is preserved. This point will be; discussed further.

308 Consider the relation of Eq. 3. 16, namely r0 rl... r 0 0 f el 0 r r1... rn 0... 0 f20... r0 r... rn 0...O.. r0 r l... rn f e (C. 6) or alternately n n r. f+ r = 1,...,q-n (C.7) i=0 i=;O Observe that any one equation out of the set given by Eq. C. 7 relates (n+l) consecutive components of f and e e Uq. Hence, it suffices to say that the v-th equation of Eq. C. 7 represents the relation between the projections of f and e in Uq onto a particular (n+1)-dimensional Un+l contains only the reference subspace, U, of where contains only the (n+l) components of the vectors which related by the v-th equation of of Eq. C. 7. It stands to reason that if m > n+l, then any (m-n) consecutive equations out of the set given by Eq. C. 7 represent the relation between the projection of the vectors f and I in some particular m-dimensional reference subspace Uk. To illustrate this let us

309 consider the m-dimensional subspaces U, g =,...,q-m, defined by mapping I: U Uq Um where [I] = [ +1 +m- =,...,q-m (C.8) and where.j denotes a q-dimensional vector whose j-th component equals one and all the other components are equal to zero. Hence, in each Um the projections of f and e in U are given by f E f() = [I Tf = and e() = [IT -[l - M ad-. (C.9) From Eq. C. 7 the relation between the vectors f( ) and () in Um becomes [A(L)(r)]T f(L) [A()(r)]T ( (C. 10) where [A(A)(r)] is an m x (m-n) matrix given by

310 (m-n) r0 0... 0 [A()(r)] = rr m r' m 0* 0 0.. r (C. 11) It should be noted that Eq. C. 10 represents Prony's formulation of the relation f() = [Z()(z)] +( = l,...,q-m (C. 12) where [Z)(z)] is an m x n matrix given by IL-1 z~-1 j.1 Mn [Z (z)] zI-m-1 M-m-1

311 Furthermore, by using the relation between the vectors r and z 3 given by Definition 3. 1, it can be shown, that the column spaces of [Z()(z)] and [A(M)(r)] are orthogonal complements in U. Up to this point we have considered the m-dimensional subspaces { g: ~L = 1,...,q-m} which contain m-consecutive components of the vectors in Uq. Let us now obtain the matrix [A(A)(r)] in the subspaces {Um: = q-m+1,...,()}, which do not contain m-consecutive components of the vectors in Uq. Here the matrix [A(k)(r)] is obtained by taking the linear combinations of the columns of the matrix [R(r)]. To show this let us observe that the matrix [A(k)(r)] given by Eq. C. 11 has been obtained from Eq. C. 6 (or Eq. C. 7) by selecting a set of (m-n)-independent rows of the matrix [R(r)]T which relate only4 the m-components of the vectors in Uq that are in U. Hence, to determine the general m x (m-n) matrix [A(k)(r)] one must obtain a set of (m-n) independent rows of [R(r)]T (or columns of [R(r)]) which relate only the m-components of the vectors in Uq that are in the subspace Uk The following example illustrates the method of forming the m x (m-n) matrix [A(k)(r)] from the matrix [R(r)]. This will be done in Lemma C. 1. 4Note that the elements of these (m-n) rows of the matrix [R(r)] T, which correspond to the (q-m) components of the vectors in Uf that are not in Um are equal to zero. 4LL'

312 Example: Let q = 9, n = 3, and m = 6. Equation C. 6 becomes r0 r1 r2 r3 0 0 0 0 0 f 0 r0 r1 r2 r3 0 0 0 0 f2 2 0 0 r r r r 0 00 f [R(r)] (f- )= 01 3 30 0 0 0r r 1 r2 r3 0 0 f4 0 O 0 0 r r2 r3 0 f5 = 0 0^^^ ^0 02 3'6 0 ~ ~ ~ ~ r0 rl r2 r3 f6- e6 f7 - 7 f- 8 8 8 f -e (C. 13) Consider the subspace U16 which contains the components of the vec9 tors f and E c U, given by the vectors f3 3 f() and (1) 5'5 f6 E6 Lf8J 1c8 (C. 14) Find the 6 x 3 matrix [A(l)(r)] which relates the vectors f(l) and e( by [A(l)(r)]T f(l) = [A(l)(r)] T E(1) (C. 15)

313 Let us post-multiply the matrix [R(r)], given in Eq. C. 13, by the 6 x 6 matrix [Y], given by r0 000 0 0 -r 1 0 0 0 0 [y 0 0 1 0 0 0 0 0 0 1 -r2 0 0 0 0 0 r3 0 0 0 0 00 1 This yields r0 0 00 0 0 0 r0 00 0 0 (r0 2-r12) 1 r0 0 0 0 (r0rrr2) r r1 r -r r 0 (03 - r2)2 0 r 2 [R(r)] [Y] -rlr3 r r1 (r0r3-r r2) 0 0 0 r3 r2 (rlr3-r22) r0 0 0 0 r3 0 r1 0 0 0 0 r r 0 0 00 0 r3 Note that the 1st, 3rd, and 5th columns of the above matrix [R(r)] [Y] relate only the components of f and c E U given by Eq. C. 14. Using these three columns, the matrix [A (1)(r)] of Eq. C. 15 becomes

314 r 0 0 (r0r2 -1 r 0 0 [A ] (rr3 r1r rr r0r2 [A1r3 r2 (r0r3-r1r2) O r3 (r r3 -r2 ) 0 0 r3 Now that the preliminary ideas have been discussed let us show that in the m-dimensional subspace U, where k e { =,...,()} the relation [A(k)(r) T f(k) [A(k)(rT (k) (C. 16) represents Eq. C. 5 in terms of "Prony's Extended Method. " We begin by making the following definitions: Definition C. 1: Let [Z(z)] be the q x n matrix defined by Eq. 1. 18 (or Eq. 1. 18a), and let [Zk(z)] by a q x n matrix defined by Z(Z) m [zkz)] — ()() A (k) (z (q-m (C. 17) 5Note that the matrix [Zk(Z)] represents the rearrangement of the rows of the matrix [Z(z)].

315 where [Z(k)(z)] is the (m x n) matrix defined by Eq. C. 3, m > n+l. Then the q x q nonsingular matrix representing the elementary row operations, which takes [Z(z)] into [Zk(z)], is denoted by [Xk] and defined by [Zk(z)] = [Xk] [Z(z)] (C. 18) Definition C. 2: Let [R(r)] be the q x (q-n) matrix defined by Eq. 3. 2, and let [Xk] be the q x q nonsingular matrix defined by Eq. C. 18. Then we denote by [Rk(r)] the q x (q-n) matrix which is equivalent to [R(r)] by the transformation [Rk(r)] = [Xk] [R(r)] [Yk] (C. 19) where [Yk] is a (q-n) x (q-n) nonsingular matrix, representing column operations. The matrix [Yk] is chosen so that when [Rk(r)] is represented by m-n q-m R,() Ir(_) Im [Rk(r)] =.-..,..R..-. (C. 20) (k) (k) R21,_) R r(_) q-m then the (q-m) x (m-n) submatrix [Rk)] is a zero matrix, i.e., [R(l) = [0] (C. 21) hk(= [RkT [Z(z)] Note, fromEq. C.3, that[Z(z)] = [k] [Z(z)]. The matrix [Xk] is equivalent to the product of the elementary matrices [F ik] defined by Thrall (Ref. 24, p. 94). kL

316 Definition C. 3: Let [R1 (r)] be the (m x m-n) submatrix, where m > n+l, of the q x (q-n) matrix [Rk(r)] defined by Eq. C. 20. Then we denote by [A (k)(r)] the m x (m-n) matrix which is column equivalent to [RA)(r)] by the transformation (k() (k ).22 [Ak(r)] = [R (r)][Kk] (C.22) where [Kk] is an (m-n) x (m-n) nonsingular matrix representing the column operations. The matrix [Kk] is chosen (k) so that the matrix [A (r)] will have the form X (k) 0...0 (r) x(k)( 2) 122 ~~(k) 2,2k (k) (k) [A () (r)] ^2,1. ^2,2^.^~3,2^ (k)~ (k (k) r [n~k~)l n1,1^.m-n,m-n^ 0 (k) (r) m (k) (lr) n+2,2 m-n+1, m-n 0 0 0 X (k)(r) (C.23) m,m-n -

317 8 where x(k)(r) =. a('j) r. i,]j-''' v...u 0 n v A Let us now state the following lemma: Lemma C. 1: Let [Z(z)] be a q x n matrix, defined by Eq. 1.18 (or Eq. 1. 18a), of maximal rank n (q > 2n), and let [R(r)] be a q x (q-n) matrix, defined by Eq. 3.2, of rank (q-n). Furthermore, let [Z(k)(z)] be a (m x n) matrix, of rank n (m > n+l), which is related to the matrix [Z(z)] by Eq. C. 3; (k) and let [A (r)] be a m x (m-n) matrix, of rank (m-n) which is related to the matrix [R(r)] according to Definition C.3. (k) m Then, Cn(Z(k), the n-dimensional subspace of Uk, defined by the column space of [z(k)], and C (A(k)), the (m-n)dimensional subspace complement of U, defined by the column space of [A(k)(r)] are orthogonal complement subspaces in Uk, that is, n(Z()) C n(A()) = U (C. 24) 8 (k) (k) The nonzero elements, Xi (r), of the matrix [A (r)], represent some polynomial in the n+l-variables {r, rl,..,rn denoting the components of the vector r e.

318 where Cn((k)) I n(A(k)) n m-n if Cn(Z) ~ Cn(R) = U (C.25) n q-n where Cn(Z) CI n(R) and where Cn(Z) is the n-dimensional subspace of U defined by the column space of [Z(z)],and C (R) is the (q-n)-dimenq-n sional subspace of Uq, defined by the column space of [R(r)]. Proof: Let us consider the q x n matrix9 [Zk] which is row equivalent to the q x n matrix [Z] and defined by Eq. C. 17. Furthermore, let us consider the q x (q-n) matrix [Rk] which is equivalent to the q x (q-n) matrix [R] and defined by Eq. C. 19. Taking the matrix product [Rk] [Zk], yields T TYk - T- T [Rk] T[Zk] = [k]R] T [Xk] T [Xk] [Z] T T =[Yk] [R] [Z] (C. 26) To simplify our notation, we shall drop the parameter vector z or r when writing the matrices [Z(z)] or [R(r)].

319 T since the matrix product ] k [Xk] = [I] by definition (Ref. 24, p. 94), where [I] is a q x q identity matrix. Recall from Lemma 3.1 that Eq. C. 25 is satisfied, when the matrix product [R] [Z] = [0]. Since, by definition, the matrix [Yk] is nonsingular, then Eq. C. 25 is satisfied, if the matrix product [Rk] T[Zk], of Eq. C. 26, satisfies [Rk][Zk]= [] (C. 27) By using the partitionings of [Rk] and [Zk], which are defined in Eq. C. 20 and Eq. C. 17, respectively, one obtains the following relations from Eq. C. 27: [R I [ z + R ] [A 0 (C. 28) [Rl(k)]T[z(k)] + [R2(k)]T[A(k)]= 0 (C.28) [R(k)]T[(k) + [R22()][A] (C. 29) Since the matrix [Yk] was chosen so that the matrix [R2(k)] is a zero matrix, then Eq. C. 28 yields, [RTll ] k[z)] = [0] (C. 30) where [R11(k) is a m x (m-n) matrix defined by Eq. C. 20 and [Z(k)] is a m x n matrix defined by Eq. C. 3. Furthermore, using the relation [R,1(k)] = [A(k)] [K]- (C. 31)

320 (k) where [A ] is a m x (m-n) matrix, defined by Eq. C. 23; then Eq. C. 30 yields [KT]1 [A (k)(r) (z)] = -[0] or alternately [A)(r)]T[Z(k()] [0] (C. 32) since [KjT] is invertible. Clearly, Eq. C. 32 states that the subspaces (k) k) Cn(Z)) and C (A) are orthogonal in Uk. Since the dimension(k) (k) ality of C (Z(k)) is n and the dimensionality of Cm n(A(k)) is (m-n), (k) (k) then C (Z(k) and C (A(k)) are orthogonal complement subspaces in U m-n in Um and so Eq. C. 24 is satisfied. Thus, the lemma is proved. It should be mentioned that the hypothesis of Lemma C. 1, which is given by Eq. C. 25, can be replaced by the polynomial equation P(z) = 0, defined by Eq. 3.4 of Chapter III. This leads to the following corollary: Corollary C. 1: Equation C. 24 of Lemma C. 1 is satisfied if the components {zl, 2,..,z } of the vector z E a are the roots of the n-th order real polynomial equation n P(z) = r.z = 0 (C.33) i=O where the ordered set {r, r,..., r } defines the vector r ed?.

321 Proof: All of this corollary is contained in Lemma 3. 1, Chapter III, and Lemma C. 2. Clearly, Eq. C. 33 simply replaces the condition given by Eq. C. 24 of Lemma C. 1. The validity of such a replacement has been established by Lemma 3. 1. At this point let us examine the converse problem, namely, if Eq. C. 24 is true, is there a unique n-th order polynomial equation, P(z) = 0, which relates the vectors r and z? Recall that if Eq. C. 33 is true, i. e., the subspaces C (Z(k) and C (Ak) are orthogonal complements in Uk, then the matrices [Z(k)(z)] and [A(k))] are related by Eq. C. 32, or equivalently by Eq. C. 28 (since the matrix [R (k)] = [0] by definition). Using Eqs. C.28, C.30, C.31, and C. 32, let us relate the matrix product [A( )] [Z(k)] to the matrix T product [R] [Z] as follows: T iKk] (k)] [T(k) _ (k) T (k)] ] FT [Iy] [Rk] [Zk] [Iy]T[Yk]T[R]T [Z]= (C. 34) where [Iy] is a (q-n) x (m-n) elementary matrix which picks the first (m-n) rows of the matrix [Rk] T Premultiplying both sides of Eq. C. 34 by the matrix [Kj], yields [A(k)] T [z(k)] = [Kk] T [Iy]T[Yk]T[R] = 0 (C. 35)

322 Since the matrix product [Kk] T [Iy] T[Yk] Trepresents the elementary row operations on the matrix product [R] T[Z], which is defined by Eq. 3.8, then in carrying out the matrix multiplication [Kk] T[Iy] [k] [R] [z] one obtains Q2( (z )P(z1) Q21 (zn)P(zn) (k) (k) l(Z )P(z ) Q ( )P(z& [A(k)]T[z(k)] I = [0] (k) (k) Qmn(Z1)P(z1) Qmn(Zn)P(zn) (C. 36) where P(zj) is defined by Eq. 3. 9 and {Qi()(zj): i = 1,2,...,m-n} represents a set of N.-th order polynomial, where N. < q-n. Since 1 1 the ij-th element of the matrix product [A (k)] T[Z(k) can be represented by Qi(k)(z) P(zj), then the n-roots, {Zl,2',...,z }, which are common to all the rows of Eq. C. 36, are given by the polynomial equation P(z) = 0 (C. 37) Note that Eq. C. 37 gives a relation between the vectors r and z which is independent of the superscript k (i. e., invariant of the

323 m -dimensional reference subspace) and also the subscript i where i = 1,2,..., m-n. These results are summarized by the following lemma: Lemma C. 2: Let [Z(z)] be a q x n matrix, defined by Eq. 1. 18 (or Eq. 1. 18a), of maximal rank n (q > 2n), and let [R(r)] be a q x (q-n) matrix, defined by Eq. 3.2, of rank (q-n). Furthermore, let [Z(k)(z)] be a m x n matrix (m > n+l), of rank n, which is related to the matrix [Z(z)] by Eq. C. 3 and let [A(k)(r)] be a m x (m-n) matrix, of rank (m-n), which is related to the matrix [R(r)] according to Definition C.3. Then, Cn(Z) Cqn(R)= Uq (C. 38) where C (Z) I C (R) n q-n if (1) C ()) 0 C (A( ) = (C.39) where Cn(Z(k)) I C n(A(k)), and if (2) {Q~(k)(z) 0 O:i = 1,2,...,m-n}

324 when (k)(z) (z) (C. 40) where {Qi(k) i = 1,2,...,m-n} is defined by Eq. C. 36. The proof of the lemma follows directly from the discussion that was presented above. We are now in a position to formulate "Prony's Extended Method" in each m -dimensional reference subspace Uk, where k e {II = 1,2,...,(m)}. Recall that Lemma C.1 tells us that for each r eg we can define a vector [A(k)r)] e Um which is aUk is orthogonal to the vector [Z(k)(z)] e U, that is ([A()(r)]i, [Zk)(z)] ) 0 (C. 41) for all e 9z and I e E n. Let us take the inner product of both sides of Eq. C. 5 with respect tothe vector [A(k)(r)]. This yields [A(k)(r)], f(k) = [A(k)(r)] e(k).42) ([A r)]i)-[A W)]i, ) (C. 42) when using the relation of Eq. C. 41. For Eq. C. 42 to hold for all e E, we obtain the relation [A(k)(r)]Tf(k)= [A(k)(r) (k)) (C.43) [A (r) -_ -_A()] () (C. 43)

325 where fk) is a prescribed realvector in Uk, and where r e and e (k)) e Uk are unknown vectors, k e {/ = 1,2,...,(q)}. This equation defines the basic relation which must be satisfied when using "Prony's Extended Method" to solve tie Chebyshev approximation problem in Uk, k e { = 1,2,...,(q)}, defined by Eq. C. 5. The following theorem shows that for every r e E and (k) Uk satisfying Eq. C. 43, one can determine a unique vector e e Uq which satisfies the relation [R(r)] T f = [R(r)] T: Theorem C. 1: Let f be a prescribed real vector in Uq. Then for each r e. and (k e Um satisfying [A(k)) T f(k) [A(k) T (k) (C44) [A (r)] - [A (r)] E (C.44) k e {, = 1,2,...,(q)}, where f(k) e U isrelatedto f Uq by Eq. C.2, and where [A(k)(r)] is defined by Eq. C. 23; there exists a unique vector e c Uq which satisfies the relation T T [R(r)] f = [R(r)] (C. 45) where [R(r)] is defined by Eq. 3. 2. Strictly speaking the vector e ()( should be denoted by (k)(, r) since it represents the vector c(k)(l,z) of Eq. C. 5.

326 Proof: Consider any index k e {c = 1,2,...,()}. Recall from Definition C. 3 that the (m) x (m-n) matrix [A()(r)] is column equivalent to the m x (m-n) matrix [R1(k)()], that is, [Ak)(r)] [R11 )(r)] [Kkr (C. 46) Furthermore, from Definition C. 2 the matrix [Rl (k)] is a submatrix of the qx (q-n) matrix [Rk(r)] defined by Eq. C. 20. Let us now represent the relation of Eq. C. 45 in terms of the matrix [Rk(r)] by using Eq. C. 19. This yields [Rk(r)] [Xk] f = [Rk(r)] [Xk] ~ (C. 47) By using Definition C. 1, the vectors [Xk] f and [Xk] e may be represented by f(k) e(k) [X]kf = [[~1] e [Xk —— I where f(k) and e represent the projections of f and e in U onto m Uk, respectively. Hence, Eq. C. 47 becomes f(k) (k) f':'J f (k) " (k) which yields the following two equations, when using the partitioning of the matrix [Rk(r)], defined by Eq. C. 20:

327 [R (k) Tf(k) = [R(k)(r)] T ( k) (C.48) [R _)] f[R (C. 48) and [R12(r)] f( + [R22(r)]T f) [R12(r)]e + [R22(r)] (k) (C. 49) (k) First we note that the vectors r and which satisfy Eq. C. 44 will also satisfy Eq. C. 48 since the matrices [A( )(r)] and [R11 )(r)] are equivalent (see Eq. C. 46). Next we note that when the vectors r and (k) are known then the only unknowns in Eq. C. 49 are given by the vector e k) since the vectors f(k) and f(k) are prescribed. Since the (q-m) x (q-m) matrix [R2(k)(r)] is nonsingular for all r 0 e., (k) then from Eq. C. 49 the (q-m)- dimensional unknown vector e ) is given by ek) (k) (r ) T] - 1 (kT (k) (k) = + [[R22)(r)] [R12 )T(fk) k)) (C.50) Using the relation (k) ]=- [X-] I it is seen that the vector E e Uq which satisfies Eq. C. 45 is fully determined when the vectors r and (k) are known and satisfy Eq. C. 44. Thus the theorem is proved.

328 An alternate way of proving the above result is to note that for a given pair of vectors r e and (k) c U, which satisfy Eq. C. 44, one can first obtain the vector pair (j,z) from Eqs. C. 33 and C. 5. Then obtain the vector e e Uq by using Eq. 4. 47 of Chapter IV. Specifically, knowing the vector r e, one can obtain the vector z c from the roots of the polynomial equation of Eq. C. 33. Then the vector _ can be obtained from the relation f(k) [z(k)(z)]1 + () in Uk since the vectors f(k) (k) (k) since the vectors f() and E are known and the matrix [Z )(z)] is fully determined by using the above calculated value of z. Once the vector pair (i,z) e Sz x a is known, then the error vector e e Uq can be determined from the relation of Eq. 4. 47 of Chapter IV. Clearly the advantage of using the method, outlined in the proof of Theorem C.1, to obtain the vector E E Uq from each estimate of r e.R and e (k) Uk m is that one need not determined the value of z 1That such a vector pair exists has been shown in Lemma C. 2.

LIST OF REFERENCES 1. P. R. Aigrain and E. M. Williams, "Synthesis of n-Reactance Networks for Desired Transient Response, " J. of Applied Physics, Vol. 20. 6, June 1949, pp. 597-600.0 2. N. I. Akhiezer, Theory of Approximation, Frederick Ungar Publishing Co., New York, 1956. 3. J. D. Brule, "Improving the Approximation to a Prescribed Time Response, "IRE Trans. on Circuit Theory, Vol. CT-6, December 1959, pp. 355-361. 4. B. M. Brown, The Mathematical Theory of Linear Systems John Wiley and Sons, Inc., New York, 1961, pp. 149-152. 5. J. W. Carr, III, "An Analytic Investigation of Transient Synthesis by Exponentials, M. S. Thesis, M. I. T., 1949. 6. B. Friedman, Principles and Techniques of Applied Mathematics, John Wiley and Sons, Inc., New York, 1956, p. 18. 7. T. N. E. Greville, "The Pseudoinverse of a Rectangular or Singular Matrix and Its Application to the Solution of Systems of Linear Equations," SIAM Review, Vol. 1, No. 1, January 1959, pp. 38-43. 8. W. H. Huggins, "Signal Theory, " IRE Trans. on Circuit Theory, Vol. CT-3, No. 4, December 1956, pp. 210-216. 9. E. I. Jury, "A Simplified Stability Criterion for Linear Discrete Systems, Proc. IRE, Vol. 50, No. 6, June 1962, pp. 1493-1500. 10. W. H. Kautz, "Transients Synthesis in the Time Domain," IRE Trans. on Circuit Theory, Vol. CT-1, No. 3, September 1954, pp. 29-39. 329

330 REFERENCES (Cont.) 11. A. N. Kolmogorov and S. V. Fomin, Elements of the Theory of Functions and Functional Analysis: Vol. I, Metric and Normed Spaces, Graylock Press, Rochester, N. Y., 1957. 12. R. M. Lerner,'The Representation of Signals,' Trans. of the 1959 International Symposium on Circuit and Information Theory, June 16-18, 1959, pp. 197-216. 13. R. N. McDonough, "Representation and Analysis of Signals: Part XV, Matched Exponents for the Representation of Signals," Dept. of Electrical Engineering, The Johns Hopkins University, April 30, 1963. 14. C. de la Vallee Poussin, "Sur laMethode de l'approximation Minimum, Soc. Sci. de Bruxelles, Annales, Seconde Partie, Memoires 35, 1911, pp. 1-16. 15. R. Prony, "Essai Experimental et Analytique sur les lois de la Dilatabilite des Fluides Elastiques, et sur Celles de la Force Expansive de la Vapeur.de L'eau et de la Vapeur de L'alkool, a Differentes Temperatures, J. de LEcole Polytechnique Vol. 1, Cahier 2, An. m, Paris, 1795, pp. 24-76. 16. H. Radamacher and I. J. Schoenberg, "Helley's Theorem on Convex Domains and Tchebyscheff's Approximation Problem, " Canadian J. of Math., 2, 1950, pp. 245-256. 17. E. Ya Remez, General Computational Method of Chebyshev Approximations. The Problem with Linear Real Parameters. Izdatel'stvo Akademia Nauk Ukrainskoii, S. S.R., Kiev, 1957. Translation: Document No. AEC-tr-:4491, Office of Technical Services, Department of Commerce, Washington 25, D. C. 18. T. J. Rivlin, "Overdetermined System of Linear Equations, " SIAM Review, Vol. 5, No. 1, January 1963, pp. 52-66. 19. J. R. Rice, The Approximation of Functions: Vol. 1, Linear Theory, Addison-Wesley Publishing Co., Inc., Reading, Mass., 1964.

331 REFERENCES (Cont.) 20. H. Ruston, "Synthesis of R-L-C Networks by Discrete Tschebyscheff Approximations in the Time Domain, " Technical Report No. 107, Electronic Defense Group, Dept. of Electrical Engineering, University:of Michigan, Ann Arbor, Michigan, April. 1960. -- 21. G. F. Simmons, Introduction to Topology and Modern Analysis, McGraw-Hill Book Co., New York, 1963. 22. E. Stiefel, "Uber Discrete and Lineare Tschebyscheff Approximationen," Numerishe Mathematik, Band 1, Springer Verlag 1959, pp. 1-28. 23. D. T. Tang,"Tschebyscheff Approximation of a Prescribed Impulse Response with R.C Network Realization," 1961 IRE International Convention Record, Vol. 9, Part 4, pp. 214-220. 24. R. M. Thrall and L. Tornkeim, Vector Spaces and Matrices, John Wiley and Sons, Inc., London, 1957. 25. D. F. Tuttle, Jr., "Network Synthesis for Prescribed Transient Response," D.- Sc. Dissertation, M. I.T., 1949. 26. W. C. Yengst, "Approximation to a Specified Time Response," IRE Trans. on Circuit Theory, Vol. CT-9, No. 2, June 1962, pp. 152-162.. S 27. T. Y. Young and W. H. Huggins, Complementary Signals and Orthogonalized Exponentials," IRE Trans. on Circuit Theory, Vol. CT-9, No. 4, December 1962, pp. 362-370. 28. L. A. Zadeh and C. A. Desoer, Linear System Theory, The State Space Approach, McGraw-Hill Book Co., New York, 1963, pp. 577-581.

DISTRIBUTION LIST No. of Copies 23 Defense Documentation Center Attn: DDC- IRS Cameron Station (Bldg. 5) Alexandria, Virginia 22314 I~1... Office of Assistant Secretary of Defense (Research and Engineering) Attn: Technical Library, RM 3E1065 Washington, D. C. 20301 1 Bureau of Ships Technical Library Attn: Code 312 Main Navy Building, Room 1528 Washington, D. C. 20325 1 - Director U. S. Naval Research Laboratory Attn: Code 2027 Washington, D. C. 20390 1 Commanding Officer and Director U. S. Navy Electronics Laboratory Attn: Library San Diego, California 92101 2 Headquarters, U. S. Air Force Attn: AFCIN Washington, D. C. 20325 1 Rome Air Development Center Attn: EMLAL- 1 Griffiss Air Force Base New York 13442 I~1 ~Headquarters Strategic Air Command Attn: DOCE Offutt Air Force Base, Nebraska 68113 332

333 DISTRIBUTION LIST (Cont.) No. of Copies 1 Systems Engineering Group (SEPIR) Wright-Patterson Air Force Base, Ohio 45433 2 Electronic Systems Division (AFSC) Scientific and Technical Info Div (ESTI) L. G. Hanscom Field Bedford, Massachusetts 01731 2 Air Force Cambridge Research Laboratories Attn: CRXL-R L. G. Hanscom Field Bedford, Massachusetts 01731 2 Chief of Research and Development Department of the Army Washington, D. C. 20315 I~1 ~Ofc of the Chief of Communications-Electronics Attn: OCC-E Department of the Army Washington, D. C. 20315 2 Commanding General U. S. Army Materiel Command Attn: R&D Directorate Washington, D. C. 20315 2 Redstone Scientific Information Center Redstone Arsenal, Alabama 35809 I ~:1 Commanding Officer 52d USASASOC Fort Huachuca, Arizona 85613 I~1 ~Commanding General U. S. Army Combat Developments Command Attn: CDCMR-E Fort Belvoir, Virginia 22060

334 DISTRIBUTION LIST (Cont. ) No. of Copies 1..Deputy Commander U. S. Army Conbat Developments Command Communications- Electronics Agency Fort Huachuca, Arizona 85613 I~1 ~Commanding Officer Harry Diamond Laboratories Connecticut Ave. and Van Ness St., N. W. Washington, D. C-. 20438 1 USAECOM Commodity Management Office AMSEL-EW Fort Monmouth, New Jersey 1 o Commanding General U. S. Army Electronics Command Attn: AMSEL-RD- LNR Fort Monmouth, New Jersey 07703 I~~~ Commanding Officer U. S. Army Electronics Command R&D Activity White Sands Missile Range, New Mexico 88002 I~1 ~Chief, Mountain View Office Electronic Warfare Lab., USAECOM P. O. Box 205 Mountain View, California 94042 2 Commanding General U. S. Army Security Agency Attn: IADEV Arlington Hall Station Arlington, Virginia 22212 1 Commanding General U. S. Army Security Agency.Attn: -IACON Arlington Hall Station Arlington, Virginia 22212

335 DISTRIBUTION LIST (Cont.) No. of Copies 1 Commanding General U. S. Army Security Agency Combat Development Activity Arlington Hall Station Arlington, Virginia 22212 1 Commanding General U. S. Army Electronics Command Attn: AMSEL-WL-D Fort Monmouth, New Jersey 07703 1''Commanding General U. S. Army Electronics Command Attn: -AMSEL-WL- C Fort Monmouth, New Jersey 07703 1 Commanding General U. S. Army Electronics Command Attn: AMSEL-WL-S Fort Monmouth, New Jersey 07703 1 Commanding General U. S. Army Electronics Command Attn: AMSEL-WL-N Fort Monmouth, New Jersey 07703 I~1 ~Commanding General U. S. Army Electronics Command Attn: AMSEL-WL-E Fort Monmouth, New Jersey 07703 I~1 ~Commanding General U. S. Army Electronics Command Attn: AMSEL-RD-GFR Fort Monmouth, New Jersey 07703 1 Commanding General U. S. Army Electronics Command Attn: AMSEL-NL-E (Mr. Reindl) Fort Monmouth, New Jersey 07703

336 DISTRIBUTION LIST (Cont. ) No. of Copies I~1 ~Commanding General U. S. Army Electronics Command Attn: AMSEL-NL-E (Mr. Hecker) Fort Monmouth, New Jersey 07703 I~1 ~Commanding General U. S. Army Electronics Command Attn: AMSEL-NL-E (Mr. Tepper) Fort Monmouth, New Jersey 07703 10 Cooley Electronics Laboratory The University of Michigan Ann Arbor, Michigan 1 Stanford Electronics Laboratories Stanford University Stanford, California

ERRATA Page xvi: Definition of'I" should be "the set of all parameter vectors r e E with 11 = 1 (See Definition 3. 3) 102. " Page xxvi, line 3: Replace word "optimum" by the word "unknown." Page 3, line 8: add "dt" in the integral. Page 5, line 10: (a, s) should be (a,s). Page 6, line 4 from bottom: replace word "order" by the word "ordered. " Page 9, line 3 from bottom: replace the word "repeated" by "repeated real." Page 13, line 15: "Eq. 1. 19" should be "Eq. 1. 9." Page 20, line 1: The elements of the second column should be multiplied by At. The elements of the third column should be multiplied by (At)j. Page 20, line 4: Replace the word "identical" by the word "identical and real. Page 53, line 4: Replace the word "fector" by the word "vector. " Page 99, bottom line: delete the words "and only if." Page 121, Footnote: add sentence "Clearly, if there is no point where v / 0, then there is no finite z. " Page 139, line 1: Ad should bed.

ERRATA - Continued Page 175, Footnote: should beb. Page 307, Footnote:, should be 1. Page 328, Footnote: Replace footnote by "Note that such a vector pair exists if v e (See Lemma C. 2). "

UNCLASSIFIED Security Classification.' DOCUMENT CONTROL DATA- R&D (Security claaaificatlon of title, body ofabatract and indexing annotation must be entered when the overall report ie classified) t. ORIGINATING ACTIVITY (Corporate author). REPORT SECURITY C LASSIFICATON The University of Michigan Unclassified Cooley Electronics Laboratory 2b. GROUP Ann Arbor, Michigan 48105 3. REPORT TITLE On the Best Chebyshev Approximation of an Impulse Response Function at a Finite Set of Equally-Spaced Points 4. DESCRIPTIVE NOTES (Type of report and inclusive datee) Technical Report (Doctoral Thesis) 5. AUTHOR(S) (Last name, firet name, Initial) Fischl, Robert 6. REPORT DATE 7a. TOTAL NO. OF PAGES 7b. NO. OF REF ~ May 1966 351 28 8a. CONTRACT OR.GRANT NO. ga. ORIGINATOR'S REPORT NUMBER(S) DA 28-043 AMC-01870(E) b. PROJECT NO. 280-M6-00287(E) 7695-2-T c. 9 b. OTHER REPORT NO(S) (Any othernumber that may be saalned thia report) d.: |TR-172 1 AVA ILABILITY/LIMTATION NOTICES This document is subject to special export controls and each transmittal to foreign governments or nationals may be made only with prior a proval of CG, U. S. Army Electronics Command, Fort Monmouth, N. J., ttn: MSEL-WL-N. 11. SUPPLEMENTARY NOTES 12. SPONSORING MILITARY ACTIVITY U. S. Army Electronics Command Fort Monmouth, New Jersey 07703 13. ABSTRACT. This study considers the approximation of a prescribed impulse response function, h(t), at a finite set of equally-spaced points of t, by a linear combination of exponential functions, so that the resulting error is minimum in the Chebyshev sense. Specifically, given a set of q value {h(ti)} of h(t), where i = tl + (i 1) At, i =,..., q; find the 2n complex constants {ak, sk}, k =1,..., n, of the function h(t) = (a1 exp[ s tl +.. + a exp [s t ) so that the Chebyshev error, lie (t) II = max Ih(ti) - h(ti)l, is minimum when q> 2n. Since the complex ak's and sk': must occur in conjugate pairs, the Laplace transform of h(t) will be a rational function of s, i.e., a network function. The time domai approximation problem is formulated in terms of an approximation in the finite dimensional vector space. Prony's method, for determining the unknown {k, Sk}. when q = 2n, is formally extended to the case when q > 2n. The existence of the best Chebyshev approximation is proved and the bounds within which lies the minimal value of lie (t.) 1 I is found. A computational algorithm, which determines the unknown ak's and sk's simultaneously, is presented. D D i JA4 1473 UNCLASSIFIED Security Classification

UNIVrHOt i Y Ur MIUnIUANI UNCLASSIFIED 3 9015 02826 7394 Security Classification 14. LINK A LINK B LINK C KEY WORDS,,,, ____ _^. _ROLE WT ROLE WT ROLE WT (1) Discrete Chebyshev Approximation (2) Exponential Approximation (3) Time Domain Network Synthesis (4) Prony's Extended Method (5) Computational Algorithm INSTRUCTIONS 1. ORIGINATING ACTIVITY: Enter the name and address imposed by security classification, using standard statements of the contractor, subcontractor, grantee, Department of De- such as: fense activity or other organization (corporate author) issuing (1) "Qualified requesters may obtain copies of this the report. report from DDC." 2a. REPORT SECURTY CLASSIFICATION: Enter the over- (2) "Foreign announcement and dissemination of this all security classification of the report. Indicate whether report by DDC is not authorized "Restricted Data" is included. Marking is to be in accordance with appropriate security regulations. (3) "U. S. Government agencies may obtain copies of this report directly from DDC. Other qualified DDC 2b. GROUP: Automatic downgrading is specified in DoD Di- users shall request through rective 5200. 10 and Armed Forces Industrial Manual. Enter the group number. Also, when applicable, show that optional,o markings have been used for Group 3 and Group 4 as author- (4) "U. S. military agencies may obtain copies of this ized. report directly from DDC Other qualified users 3. REPORT TITLE: Enter the complete report title in all shall request through capital letters. Titles in all cases should be unclassified. If a meaningful title cannot be selected without classifica- tion, show title classification in all capitals in parenthesis (5) "All distribution of this report is controlled. Qualimmediately following the title. ified DDC users shall request through 4. DESCRIPTIVE NOTES: If appropriate, enter the type of,__ report, e.g., interim, progress, summary, annual, or final. If the report has been furnished to the Office of Technical Give the inclusive dates when a specific reporting period is Services, Department of Commerce, for sale to the public, indicovered. cate this fact and enter the price, if known. 5. AUTHOR(S): Enter the name(s) of author(s) as shown on 11. SUPPLEMENTARY NOTES: Use for additional explanaor in the report. Enter last name, first name, middle initial, tory notes. If military, show rank and branch of service. The name of the principal.athor is an absolute minimum requirement. 12. SPONSORING MILITARY ACTIVITY: Enter the name of the departmental project office or laboratory sponsoring (pay6. REPORT DATE: Enter the date of the report as day, in for) the research and development. Include address. month, year; or month, year. If more than one date appears on the report, use date of publication. 13. ABSTRACT: Enter an abstract giving a brief and factual,. TOTAL NMEOPA Ssummary of the- document indicative of the report, even though 7a. TOTAL NUMBER OF PAGES: The total page count it may also appear elsewhere in the body of the technical reshould follow normal pagination procedures, i.e., enter the port. If additional space is required, a continuation sheet shall number of pages containing information.' be attached. 7b. NUMBER OF REFERENCES: Enter the total number of It is highly desirable that the abstract of classified reports references cited in the report. be unclassified. Each paragraph of the abstract shall end with 8a. CONTRACT OR GRANT NUMBER: If appropriate, enter an indication of the military security classification of the inthe applicable number of the contract or grant under which formation in the paragraph, represented as (TS), (S), (C), or (U). the report was written. There is no limitation on the length of the abstract. How8b, 8c, & 8d. PROJECT NUMBER: Enter the appropriate ever, the suggested length is from 150 to 225 words. military department identification, such as project number, subproject number system numbers, task number, etc. 14. KEY WORDS: Key words are technically meaningful terms subproject numbe ste taetor short phrases that characterize a report and may be used as 9a. ORIGINATOR'S REPORT NUMBER(S): Enter the offi- index entries for cataloging the report. Key words must be cial report number by which the document will be identified selected so that no security classification is required. Identiand controlled by the originating activity. This number must fiers, such as equipment model designation, trade name, military be unique to this report. project code name, geographic location, may be used as key 9b. OTHER REPORT NUMBER(S): If the report has been words but will be followed by an indication of technical conassigned any other repcrt numbers (either by the originator text. The assignment of links, rules, and weights is optional. or by the sponsor), also enter this number(s). 10. AVAILABILITY/LIMITATION NOTICES: Enter any limitations on further dissemination of the report, other than those UNCLASSIFIED