OPERATING CHARACTERISTICS FOR DETECTION OF PROCESS CHANGE PART 2: COMPUTATION OF PERFORMANCE MEASURES Stephen M. Pollock Department of Industrial and Operations Engineering The University of Michigan Ann Arbor, MI 48109-2117 and Jeffrey M. Alden GM NAO R&D Center Warren, MI 48090 Technical Report 93-33 November 1993

1. Introduction In this paper we explore the numerical computation of the performance measures associated with the monitoring policy described in Pollock and Alden (1992), hereafter referred to as P&A. In particular, we present a numerical approximation method to solve equations (A3), (A4), and (A5) in P&A. As shown in P&A, this is equivalent to finding steady state solutions for the state probabilities of the approximating Markov chain MCZ of Appendix B. 2. Computing Performance Measures given P. We recall, from Appendix B of P&A, that the state space of MCZ is {0, 1,2,...2m}. The "renewal" state is {0}, the false alarm state is {m}, the true alarm state is {2m}, the set { 1, 2,...,m - 1} represents states where the system is in condition G, and the set {m + 1, m + 2,... 2m - 1} represents states where the system is in condition B. Ne first assume that the (2m + 1) x (2m + 1) transition probability matrix P is available, w le re [P]ij = prob. {Zn+1 = j Zn = i} ij = 0,1,2,...2m, and Zn is the ilCZ state after observating X'n. Sections [3] and [6] provide the elements of P for Bernoulli and Normal observations, respectively. The steady state probability vector it, where E (o0 T1... 72m), is then obtained from the Chapman-Kolmogorov equation irt = P (1) 2

along with the "normalizing" equation 2m E - 1. (2) i=o P can be expressed in terms of a 2m x 2m matrix P, a 2m-dimensional row vector po and a 2m-dimensional column vector pi: O P 1 (3) and ir can be written in terms of ro and the 2m-dimensional row vector Xr = (7r1 7r... 7r2m) so that r = (7ro r). (Recall ro= -steady-state probability the system occupies the renewal state. This in turn equals the checking rate, since the renewal state sojourn time is assumed to be 1.) This allows equations (1) and (2) to be written 0 Po (ro r) = (T7o 7r) (4) Pi PP and 7o + rl = 1 (5) where 1 (1 1 1... 1)' is the unit 2m-dimensional column vector. E(quation (4) can then be written as the equation 7o = 7p, (6) anl( the set of equations r = rroPo + 7rP. (7) 3

From the development in P&A and the definition of Pi, we know that p, has elements of 1 in the mth and 2mth rows only, and is zero everywhere else. Thus equation (6) reduces to 7rO = ITm + 7r2m = 7rG + rB where Tr- = steady state probability system is in the "true alarm" state, and 7B = steady state probability system is in the "false alarm" state. As shown in P&A, the structure of the transition matrix P allows it to be further decomposed as. (1 -a)G aB \ P = ~i-\ Add | (8),0 B where: a = probability of transition from condition G to condition B in a single time period; G is the m x m matrix, with elements Gij describing Zn-transitions while the system is in condition G, and B is the m x m matrix, with elements Bij, describing Zn-transitions while the system is in condition B. Note the m'h row of both G and B contains all zeros, since the next state from an alarm state must be the renewal state. We can represent the row vector po, which represents probabilities of transition from the renewal state to all other states, by Po = [(-a)g ab] (9) where g and b are m-dimensional row vectors with components gi and bi. Similarly, the steady state vector 7r can be written 7 = [=rg rb]. (10) 4

With these representations, equations (7) become (1-Ca) G aB [7 7rb]= 7ro[(l - a)g ab] + [7rg 7rb] (11) L 0 B or the two sets of linear equations: 7rg = (1 - a)7rog + (1 - a)7rgG (12) and Trb = a7rob + airgB + -rbB. (13) The normalization condition of equation (5) can be re-stated as: 7ro + 7rgl + -rbl = 1. (14) The method of computation begins by letting rg = 7rg/7ro and *b = 7b/7ro; then equations (12), (13) and (14) become *g[I - (1 -a)G]= (1-a)g (15) lTb[I-B] =ab + argB (16) 7ro =. (1 +.4 l +4bl)-1. (17) Equation (15) is readily solved for *,, since it is a set of m linear equations in m unknowns. Once ir9 is obtained, equation (16) can be solved for rb - another set of m linear equations. Finally, ro is obtained from equation (17), which gives g = -gTr7o (18) 76 = TrbTO. (19) Some performance measures of interest are 5

ro = checking rate, obtained from (17) 7r* = false alarm rate: the mth element of lrg obtained from equation (18) PB = prob.{system is in condition B} = Trbl found by summing the elements in 7rb obtained from equation (19). 3. Elements of P for Bernoulli monitoring In the case of Bernoulli monitoring, we observe random variables X, = 0 or 1, with distribution p(x) when in condition G given by 1 -a if x= O p(x) = a if x=1 and distribution q(x) when in condition B given by /3 if x =0 q(x)= < 1 -/ if x=1. As shown in Section 9 of P&A, by defining wo0 (1-a)(l-a) w = l-aa the evolution of the monitored Zn process is governed by wo(Z + 1) if X+1 = (20) Zn+l = — (20) wl(Zn + 1) if X+1 = 1. Selecting the value of m, which determines the dimension of G, g, B and b, is related to the desired accuracy of the computation of performance measures. As noted in P&A, the number of possible states occupied by the Zn process (MPZ) generated by equation (20) 6

is generally unbounded. Selecting a value of m is equivalent to determining the degree to which the discrete state approximation (MCZ) represents MPZ. For the moment, we assume m is given (determining a value for m is left to section 4). The key issues in determining the elements of G, g, B and b are a) identifying a set of z-values S - {z:i = 0,1,..., m}, where zo = 0 and Zm = z*; b) associating each Zn-value generated by equation (20) with an element in S. We use an arbitrary but simple method for item b): assign Z,n+ to the element Zk E S that is closest in absolute value. This is expressed mathematically as Jo(i) = arg min {l Wo(z + 1)-zk } k=O,1,...,m J(i) = arg min {IWl(Zi + 1)-zk |} k=0,1...,m The elements of the matrices G and B needed for solving equation (15), (16), and (17) are (1 - a) ifj=Jo(O) j = a if j = J1(0) 0 other j (1-a) ifj=Jo(i), i= 1,2,...m-1 a if j Jl(i), i=1,2,...m-1 0 other j, i = 1,2,...m -1 0 all j, i=m 7

[ if j = Jo(0) bj=. 1-/ if j = i() 0 other j /3 if =Jo(i), i =,2,...m-1 1-/3 if j = Jl(i), i =1,2,...m-1 0 other j, i = 1,2,...m - 1 0 all j, i = m. 4. Algorithm for defining {zi} Defining the subset {Zl, z2... Zm-1} of S is somewhat arbitrary. A simple algorithm that seems to provide numerically stable performance measures is to generate all possible ~-vallies over a "horizon" of h observations of X. The advantages of this approach are: * only feasible z-values are generated, * it accurately represents the Z, process over the horizon, and * the value of n, and consequently the accuracy of the approximation, can be increased by increasing h (see Section [5]). The algorithm follows: Algorithm for generating m and S- {'i}=O 1. Set So = {0,z'}. 8

2. Select a horizon h > 1. 3. Set n = 1. 4. Generate so = {wo(zi + 1): zi E Sn,- and wo(zi + 1) < z*}. 5. Generate sl = {wl(zi + 1): zi S-_1 and wi(zi + 1) < z*}. 6. Set Sn = {z: z E o U s1 and z Un-loSi}. 7. Increment n by 1. 8. If n < h then go to step 4. 9. Set S = UioSi and set m = ISI - 1. At termination, the set S will be a set of m + 1 distinct z-values which, when sorted by increasing value, can be labeled ZO, Z1, z2,..., Z.m Thus, given values of a, 3, a, and z*, as well as the horizon h, the algorithm provides a value of m and set S. A larger h generally produces a larger m. It is easy to show that m < 2h. Our computational experience has often shown that in increases close to exponentially in h. 5. Elements of P for Normal monitoring. In the case of Normal monitoring, the Xn are independent normally distributed random variables with distribution parameters depending only on system condition. Without loss of generality, we assume the distribution of Xn is p(x) with mean = 0 and standard deviation = 1 while in condition G, and is q(x) with mean = y and standard deviation = 1 while in condition B3: lr1 __- 2/2 p(x) = ex 9

I _(_.-)2/ q(x) = e2 The likelihood ratio is L(x) = q(x)/p(x) = eU'-t2/2 and equation (16) of P&A becomes Zn+l = ye"Xln[1 + Zn], where e-U2/2 1-a Again, we assume the set of Z-values {zo 0, z1, Z2,... m-,, Zm z*} is given. Then, using the governing equation (20) given Zn = y, we compute the cumulative distribution of Zn+l, from which the associated pdf can be obtained: prob.{Zn+l < z} = prob.{yeAXn(1 + y) < z} = prob.{X, l < n (1 y)} When the system is in condition G, Xn has pdf p(x). Thus the elements of G are given by G,j = prob.{Zn+i = zIj Zn = Zi} = prob.{Zn+1 < zj I Zn = z;} - prob. {Zn+1 < zj-1 I Zn-= zi} l+zit) -(I) ln(l + zi) =( 7 ) ( n +Z 1+z i,- = 1,2,...,m-. where 4(.) is the standard normal cumulative distribution. By definition, we also have Gmj=0 j =,2,...,m. Since state Zm = z* represents an alarm state, [G], = prob.{Zn+l >* I Zn = z} = 1-( n -1) i=1,2,...,m-1. 10

Elements of g correspond to row "zero" of G, so that gj = prob.{Z,+ = Zj I Zn = 0} -= (I(lIn z -) j =1,2,...,m-1 gm = 1-()l In By a similar argument, while the system is in condition B, X, has pdf q(x), and so B- = >(In (Z+ )- ()- (l In (+z)i j = 1,2..., m-1 Bmj = 0 j=1,2,...,mBim = 1 - ( ln -z) i= 1,2,...,m- 1 1 (I+ a) -I bj = 4)( lnzi -. - ln - I) j=l,2,...,m-1 7 -y bm = 1 —(jlnz -z). 6. Computation for Normal monitoring Exercising equations (15) through (17) for the Normal monitoring case requires a set S = {- z }o such that the resulting linear equations have numerically stable solutions as ni becomes large. Because of the peculiar nature of the matrices G and B, defined in the preceding section, the task of generating an appropriate set S is difficult. Fortunately, however, Xn is a continuous random variable, which allows a re-stating of equations (15) through ( 17). This re-formulation allows their solution by means of existing techniques from numerical analysis. In particular, we define the elements of S to be iz' zi = iA =- i = 0, 1,2,...,m m so that A is the interval between equally spaced points from Z, = 0 and Zn = z*. This allows the definition of continuous analogues of the steady-state probability vectors'ir and r'b; that is, we define fg(z) and fb(z) so that, in the limit as a - 0, 11

Arog(z)^ = (steady state) prob.{z < Z < z + A n system is in condition G} 7rofb(z)A = (steady state) prob.{z < Z < z + A n system is in condition B} and so f9(zi)A = [*g]i and fb(zi) = [rb]Li. Equation (15) can then be written m [frg]j = (1- a)[irg]iGij + (1 - a)gj j = 1,2,...,m -1 (21) i=l m [irg]m = (1 -a)E[irg]Gim + (1-a)gm. (22) i=1 Using the definition of f(.), the first set of these becomes: m A1 Jf(zj)= (1- a) fg(zi)^Gi + (1 -a)gj j 1,2,...,m-1 (23) i=1 From the definitions of Gij and gj given in Section 5, it can be readily shown for small A and i,j = 1,2,...,m -1 that = ^ ( 1 In Zj ) n -tPj p I y(l + zi) and 9- yzjp In. Dividing equation (23) by A and taking the limit as A - 0 gives ( -9(a)f x)-P In dx)) +(l-a)1 -In z)0<z<z* (24) By a similar argument, fb(z) = j [fb(x) + afg(x)]zq n 1 ) dx + a q In) < z < z (25) Equations (23) and (24) are the continuous equivalents of equations (15) and (16). The solution method is essentially the same: we first solve equation (24) for fg(z), and then 12

solve equation (25) for fb(z). Substituting Gim = 1 - 7Tjj' Gij and g, = 1 - E -l gj into equation (22) and then using the continuous approximations for Gij and gj, we can derive the alarm probabilities: ['g]m = (1 -a) fg) -p( -n (l+ z)dzdx+(1-a) P (1 n dz Jo ** h" fIZ IL 7(1 + X)) iZ \IL 7/ (26) [irb]m = [fb(x) + afg(x)] j — q In dzdx + )a q (1in dz Z'Y(1 + X) Z L 7 (27) Finally, the normalizing equation equivalent to equation (17) is % 1 A iM +o 7 = 1 + fg(x)dx + fb(x)dx + [*g]m + [*b],. (28) Equations (24) and (25) are Fredholm equations of the second kind, which, as pointed out in P&A, have been examined extensively in the literature. The key to their solution is the nature of their kernals, that is the behavior of kg (, ) = zP ( Inl+) 1 2 Z - 1 e 2P2 y(1 + x) and s1 2 n z-)z 2 kb(X, z) =- - 2 -) In order to obtain a numerical solution to equation (24), a FORTRAN code was written using a NAG library subroutine (NAG, 1983) that solves this linear non-singular Fredholm integral equation of the second kind using the method of El-Gendi(1969). The function fg(.), appearing on both sides of equation (24), is approximated by truncating a Chebychev 13

series to form an nth order polynomial. The subroutine solves for the resulting polynomial coefficients c,i = 1,2,..., n such that fg(z) E 4 cos (i - 1)C 1 (C -1)) (29) i=l with the property z fg(z) dz E 1)2 (30) *^=0 i=1; i odd This analysis also provides fg(xi) evaluated over the set of Chebyshev points xi where xi =-~- + cos ~7(i - 1)) Xi = (l + cos (n)) =1,2,...,n. (31) Equation (25) is then solved, using the same NAG subroutine, by approximating fb(.) with a different Chebychev series and using fg(.) obtained above. This produces the coefficients c, i = 1,2,..., n, such that fb(z) ccos (i - 1)cosl ( -1)), (32) with the property J fb(z) dz E I (33) ==1;. odd 1 — (i — 1)2' The performance measures [Tg]m and [1b`]m are obtained by trapezodial approximation of the integrals in equations (26) and (27) using the values of fg(xi) and fb(xi) obtained from equations (29) and (32) at the Chebyshev points xi, i = 1,2,..., n. Finally, 710 is obtained Ib) sub)stituting equations (30), (33), and the values of [rg]m and [irb]m into equation (28). 7. Numerical Results for Bernoulli Monitoring In this section we present numerical results for Bernoulli monitoring. The first computationi is generating m and the set S using the algorithm of section 4. Figures 7.la, 7.1b, and 14

7.1c show how m = ISI increases with increasing horizon h, for various values of a, /, a, and p*. Comparison of these figures show how larger p*, smaller a, and larger a and 3 increase the number of states generated for a given h. 7.1 Generating S = {zi} Figures 7.2a, 72.b, and 7.2c show the resulting Zi values and associated steady-state probabilities when a = 3 = 0.2, a = 0.01, p* = 0.2. Note the "jumpy" nature of the steady-state probabilities and the "gaps" in the z-axis. This behavior suggests that a) any solution approach which uses a continuous approximation to the state space would be unwieldy, and b) a using simple "grid" over the z axis to represent the possible z values would be inefficient due to computations associated with highly unlikely or even impossible z values. Figure 7.2b suggests a possible fractal character of the zi values by showing them in the region near zero (0 < zi < 5) which produces a plot similar to Figure 7.2a. Figure 7.2c shows how increasing the horizon from 8 to 12 (which increases the number of generated states) tends to fill in more values on the z axis but, has little effect on the predonlinant state probabilities. The accuracy of the computations generally increases with h, as does the computational effort. Figure 7.3 illustrates how the increased accuracy (represented in the figure by the comnllpte(l value of pro) is unbiased and levels off near h = 7. Over a wide range of parameter settings. the maximum observed absolute percent error is less than 5 percent for h > 7. For this reason, we use h = 7 for the remaining computational results. 7.2 Operating Characteristics In production management, a trade-off is often made between time spent producing 15

scrap and time spent "down" (i.e., not producing anything). In our analysis we know that increasing p* reduces the down time due to false alarms yet this increases the time spent producing scrap. Such trade-offs can be captured by means of an operating characteristic (OC) curve (see P&A) which is created by plotting two competing performance measures as a function of a decision variable (such as p*). Figures 7.4a, 7.4b, and 7.4c each show three OC curves (for a E {0.1,0.05,0.01)) that plot PB = Pr{system is in condition B} = Pr{producing scrap) versus 7ro = checking rate = Pr{down for checking} while varying p* from 0.01 to 0.5. Figure 7.4a shows that, for a fairly non-informative sensor (a = P = 0.3), varying p* allows a wide range of operating points, with a resulting wide range of possible PB and 7ro values. With a more sensitive sensor (Figure 7.4c with a = / = 0.1), only a few operating points are possible for p* between 0.01 and 0.5. Indeed, for a = 0.1 there is only one feasible operating point in this range of p*: PB = 0.00747 and rO = 0.146. Figure 7.5 shows an OC for an "asymmetric" sensor having a = 0.3 and 3 = 0.2. Note that improving only one error probability of the sensor, i.e., / = Pr{xr = OG} from / = 0.3 (as in Figure 7.4a), improves both measures plotted in the OC. This is because a decrease in 3 reduces the time spent producing scrap by reducing the detection time. This also reduces the probability the system is down: a lower /3 gives greater confidence that x = 0 implies a good system; this reduces the upward drift in the Zn process for a given set of observations containing zeros; finally, this delays the time until a false alarm occurs for a fixed alarm threshold. Figure 7.6 compares different sensors for a system with a = 0.01. This OC allows an immediate assessment of the advantage of a more sensitive sensor. For example, if an operating policy with Pr{B} = 0.04 is required, decreasing the error probabilities of the sensor from a = 3 = 0.3 to a = 3 = 0.2 decreases 7ro from about 0.11 to about 0.05. Figure 7.7 shows an alternative form of an OC. The two attributes are PB and Pr{Producing good product) = r-o + PG. The latter measure is important since time spent checking causes 16

a decrease in production capacity even though there is less scrap produced. A fascinating result of this OC is the existance of operating points (above the dotted line) that are never optimal regardless of the (positive) cost per scrapped unit and (positive) revenue per good unit produced. For example when a = 0.01, there exits for each probability threshold p* above 0.26 a p* below 0.26 producing the same throughput of good product and a lower scrap rate. 8. Numerical Results for Normal Monitoring Figure 8.1 shows the probability density function rofg(z) for Normal monitoring. Compared to Bernoulli monitoring (Figure 7.2) this distribution is smooth and well behaved except near zero where, it can be shown that limzo fg(z) = 0. Figure 8.2 shows the Operating Characteristic Curves for PB = Pr{Producing scrap} versus 7rT = Pr{Producing good product} for two values of a (0.05 and 0.1) while fixing /l = 1.5. As in Figure 7.4a, there is an improvement in the OC with smaller a. Since a is th e exp)ected numllber of inter-monitoring intervals until system failure, a can be reduced by c it her inlcrIeasing the actual life of a machine or by decreasing the inter-monitoring interval. lFiglure S.3 shows OC sensitivity to changing /, the shift in the expected observation value wlen thie system fails. As pi increasesthe power of the sensor to discriminate between cod(ltitions G and B increases and this improves the OC curve. This has obvious implications in ecvaluating sensors with different jl values. Figulre S.4 shiows the alternative OC curves for Pr{Producing scrap} versus Pr{Producing good lpro(luct} for a = 0.05 and u e {0.5. 1.0, 1.5}. As in Figure 7.7, if this OC represents a significant trade-off, then there is a wide range of probability thresholds that can be ignored \whlien selecting an operating point. For example, when p =.5 and a = 0.05 all p* above 0.15 can be ignored. 17

References Pollock, S. M and Alden J. M., 1992, "Operating Characteristics for Detection of Process Change Part 1: Theoretical Development", Technical Report 92-34, IOE Dept., The University of Michigan. NAG Fortran Library Routine, 1983, D05ABE, February 1983. El-Gendi, S. E., 1969, "Chebyshev Solution of Differential, Integral and Integro-Differential Equations," Computer Journal 12, pp 282-287, 1969. 18

Bernoulli Monitoring: State Space Size a=0.1, alpha=beta=0.15 160 1 —-- p* = 0.7 140 - p* = 0.1 or 0.4 120 _______, 100 - / 8o 0 80 / E 60 z Z 40 20 1 2 3 4 5 6 7 8 9 10 11 12 Horizon Used to Generate State Space Figure 7.la: The number of states generated increases as the horizon used to generate the z-state space increases. 19

Bernoulli Monitoring: State Space Size a=0.01, alpha=beta=0.15 300 p* = 0.7 250 l o P * = 0.4 S, -* p* = 0.1 A 200 p =0 1 ~ 150 E 100 z 50 2 3 4 5 6 7 8 9 10 11 12 Horizon Used to Generate State Space Figure 7.1b: Increasing mean time between failures by decreasing a from 0.1 (as in Figure 7.1a) to 0.01 (above) increases the number of generated states. 20

Bernoulli Monitoring: State Space Size a=0.1, alpha=beta=0.35 1200 p* = 0.7 1000 -- p* = 0.4 80 - -- p* = 0.1 800 - 0 600 E E 400 z 00 - 2 3 4 5 6 7 8 9 10 11 12 Horizon Used to Generate State Space Figure 7.1c: Decreasing observation information content by increasing a and P from 0.15 (as in Figure 7.la) to 0.35 (above) increases the number of generated states. 21

Bernoulli Monitoring: Probability Distribution over z a=0.01, alpha=beta=0.2, p*=0.2, h=8 0.3-'O 10 0 0 0.25 - N ~ 0.2 0.1 0 ~ 0.1 - C. 5 0.05 -- - 0 5 10 15 20 25 z value Figure 7.2a: The probability mass function [Tr,]i associated with each z value generated by the solution proceedure when c = - = 0.2, a = 0.01, p* = 0.2, and a horizon of h = 8 is used to generate the state space. 22

Bernoulli Monitoring: Probability Distribution over z a=0.01, alpha=beta=0.2, p*=0.2, h=8 0.3 13 0 0 0.25N ~ 0.2 ~ 0.15-._, 0 co n 0.1 - 0 -0 ) 0.05. - - 0. -L^~- _ 0 - -—,.', -. -, -,,.,,- -,. —.' - -' | ---- I 0 1 2 3 4 5 z value Figure 7.2b: [7rg]i of Figure 7.2a in the interval 0 < z < 6 shows more detail. Note the "self similarity" of points in the interval [1.5,2] to those in the interval [5,7] of Figure 7.2a. 23

Bernoulli Monitoring: Probability Distribution over z a=0.01, alpha=beta=0.2, p*=0.2, h=12 0.3 0o o 0.250.2 X3 0.15 -.. 0 u0 0.1 0 0.05 -, L. _ 0.: 0o~- r-... - - - - I" ----- I 0 5 10 15 20 25 z value Figure 7.2c: Increasing the horizon h from 8 (as in Figure 7.2a) to 12 (above) introduces new z values, but has a minor effect on the probabilities associated with the old z values. 24

Bernoulli Monitoring: Convergence 30. 225 20 15 u) 5.' - 0 0 Z 3 -41012 0-10 -15= o 2 -20 0. -25 3 4 5 6 7 8 9 10 11 12 h = Horizon Used to Generate State Space Figure 7.3: Percent change in wr0 as the horizon used to generate the z-state space increases from h - 1 to h. The figure displays 18 curves corresponding to all possible parameter settings such that c = p3 E {0.15,0.25,0.35}, a E {0.01,0.1}, and p' E {0.1,0.4,0.7}. Six of the parameter settings had zero percent change for all h shown. The two worst cases are plotted using diamonds (a = /3 = 0.35, a = O.Ol,p* = 0.4) and triangles (a = /3 = 0.35,a = 0.01,Op = 0.1). 25

Bernoulli Monitoring: Operating Characteristic Curves alpha=beta=0.3 0.16 _ 0.14 P*=0.5 — 0 — a = 0.01. 12 X 4 ------ -a =0.05 0.14 0.1- 0 --— o —- a = 0.05 0.1 0.08 ------- 2 0.068 0 0.04 -. 0.3 0.02 - ^ -~0'o 0 0.05 0.1 0.15 0.2 0.25 0.3 Pr{System Down) Figure 7.4a: Operating Characteristic Curves under Bernoulli monitoring for Pr{Producing Scrap} versus Pr{System Down} generated by varying p* from 0.01 to 0.5 in steps of 0.01. There are three curves, one for each a E {0.01,0.05,0.1} with a =, = 0.3 in each case. Any apparent non-convexity of these curves is due to round off errors in wG and 7r* which are used to calculate Pr{producing scrap} and Pr{system down}. Points associated with the same value of p* are connected by solid lines for p" = 0.01, 0.3 and 0.5. 26

Bernoulli Monitoring: Operating Characteristic Curves alpha=beta=0.2 0.12 -- -— a = 0.01 0.1 - p*=0.5 X | / Ad | -— o —- a = 005 s 0.08" / " -— o — a= 0.1 ~0.06 -- =0.3 ~ 0.04 — -,, 0.02 - q p*=0.01 to 0.3 ~ ^oo _ ^_ _ _~' y od=00.01 to 0.17 0 -Ol 0II- - 0-=0.0j to 0.03 0 0.05 0.1 0.15 0.2 0.25 Pr{System Down} Figure 7.4b: Operating Characteristic Curves of Figure 7.4a but with a = /3 decreased from 0.3 to 0.2. Note that increasing the failure rate tends to collapse ranges of smaller p' into one operating point, for example p* E (0.01,0.3) produces a wide range of operating points when a = 0.01, but produces only one operating point when a = 0.1. Comparing this figure with Figures 7.4a and 7.4c shows this collapse is more pronounced with smaller a and 3. 27

Bernoulli Monitoring: Operating Characteristic Curves alpha=beta=0.1 0.05 p*=0.5 0.045 - --— a = 0.01 Q- 0.04 - --— o —=.CL'0 a = 0.05 0.035 cn 0 a =0.1 0.03 0.025 0.02 -. 0.015 - 0. 0.01 O, 0.01 S* =0 1p*=0.01 to 0.5 0.005 p ~ j*=0.01 to 0.32 0 QI^' =0.01 to.09 0 0.05 0.1 0.15 0.2 0.25 Pr{System Down} Figure 7.4c: Operating Characteristic Curves of Figure 7.4a with a = 3 reduced from 0.3 to 0.1. Note the extreme collapse of operating points to just one for all p* G (0.01,0.5) associated with a high failure rate (a = 0.1) and informative sensors. 28

Bernoulli Monitoring: Operating Characteristic Curves Asymmetric Sensor alpha=0.3, beta=0.2 0.16 --— o- --- a = 0.01 0.14,0-.12,- -— n-o —- a = 0.05 0.08 0.12 0.06 - 0.02- (b 0f" o102 L- ^ ^ _ - o S Z ffi 0 -______'7 " " -__________ 0 0.05 0.1 0.15 0.2 0.25 0.3 Pr{System Down) Figure 7.5: Operating Characteristic Curves of Figure 5.4A with an asymmetric sensor: a = 0.3, 0. = 0.2. Decreasing 0 from 0.3 (as in Figure 7.4a) to 0.2 (above) improves both performance measures. 29

Bernoulli Monitoring: Operating Characteristic Curves a= 0.1 0.07 0.06'T -- - -o- - - alpha=beta=.1? 0.05 co I F~ — 0 —---- alpha=beta=.2 _ 0 04 "'0c~~~ 0 ^-04 -- o — alpha=beta=.3 n 0.032 0.02 ~aa. 0.01 - _o -00- - - - _ _I,^^^~0 >"""'O''' O.. 0 0.05 0.1 0.15 0.2 0.25 Pr{System Down} Figure 7.6: Decreasing a and 3 of a symetric sensor from 0.3 to 0.1 dramatically improves the Operating Characteristic for scrap production versus system down time. 30

Bernoulli Monitoring: Operating Characteristic Curves alpha=beta=0.3 0.16 - -- -- a = 0.01 0.14 - X ------- a= 0.05 - 0.12 - ------ a= 0.1 z 0.1 0.1 - |- ^*..36 i o) o~oP b"'-*-,32 o 0.06- d. 0.04'- 8o p*=.26 X 0. 0 0.02 -- o 0 I o-0 — - -I 0.7 0.75 0.8 0.85 0.9 0.95 Pr{Producing good product} Figure 7.7: Operating Characteristic Curve for scrap production versus good production. "Better" is towards the point (1,0): no scrapping and always producing good product (nlo down time). Note the non-optimal operating points above the dotted line: when a = 0.01, for example, for each p* above 0.26 there exists a p' below 0.26 with the same throughput of good product and a lower scrap rate. Similar "optimality threshold boundaries" for a = 0.05 and a = 0.1 are near 0.32 and 0.36, respectively. 31

Normal Monitoring: Probability Density over z a=0.05, mu=.5, p*=0.2 0.35 0.3 A N N o a 0.25 o e 0 0.2 0 cn 0.05) - 015 ( L5 0 1 2 3 4 5 z value Figure 8.1: Probability density function rofg(z) over states in {z: 0 < z < z* and in condition G} given an alarm threshold of p* = 0.2 (or z* = 5). Observations X, are normally distributed random variables with a variance of 1 and a mean p that shifts from zero to 0.5 when the system fails. Failure times are geometrically distributed with rate a = 0.05. It can be shown that lim.o fg(z) = 0. 32

Normal Monitoring: Operating Characteristic Curves mu=1.5 0.06 05 ~'06 r~ ~,4 P* = 0.5 00'' %| - -— a = 0.05 b0.05 - /, "'",P' - ---- -a =0.1 on 0.04 d Q v\ Q Q' 0 0. 003 -:. 0.02' 0. ^'. 0.01' —'' 0-1 p* = 0.15 0 0.02 0.04 0.06 0.08 Pr{System Down} Figure 8.2: Operating Characteristic Curves generated by fixing # = 1.5 and varying p* from 0.15 to 0.5 in steps of 0.05. Points associated with the same value of p* are connected by a dotted line for p* = 0.15 and 0.5. 33

Normal Monitoring: Operating Characteristic Curves a=0.05 0.20.18- * = 0.5' q - ~ —--- mu = 1.5 - 0.16, 0.14-, —'-b mu= 1.0 "0.14 0.12 t ~ I O — 4 — mu = 0.5 m 0.1''b, 0 0.08 - 0.06 0 0. ~ Ob o * 0.04 -- ~ 0.4 q0^0%o p = 0.1 0.02 o - - do 0I 0 0.02 0.04 0.06 0.08 0.1 0.12 PrfSystem Down) Figure 8.3: Operating Characteristic Curves generated by fixing a = 0.05 and varying p* from 0.1 to 0.5 in steps of 0.05. There are three curves: one for each mean observation value p E {1.5, 1.0, 0.5}. Points associated with the same value of p* are connected by a dotted line for p* = 0.1 and 0.5. A higher p implies a more discrimating sensor which improves the Operating Characteristic. 34

Normal Monitoring: Operating Characteristic Curves a=0.05 0.2 - -- mu = 1.5 0.18- o \.o --— 0 —- mu = 1.0 - 0.16 - co ^\ -- -o- -- mu =.5 t 0.14- o S. 0.12 -- 0.1 b o> \ I 0.08- - 0.06 —.- 0.06 -tp. ^.L 15 b / - - -- 2 0.04 p*25 0.02 - =25 0 -- 0.75 0.8 0.85 0.9 0.95 1 Pr{Producing Good Product} Figure 8.4: Operating Characteristi ('urve under Normal monitoring for scrap production versus good pro(liuction with fixed a = 0.05. " Better" is towards the point (1,0): no scrapping and always producing good product (no down time). Note the non,-optimal operating points above the dotted line: when it = 1. for examrple, for each p* above 0.2 there exits a p* below 0.2 with thle samle throughput of good product and a lower scrap rate. Similar "optirmalit.y threshold boundaries" for fi = 0.5 and 1.5 are near p' = 0.15 and 0.25, respectively. 35

UNIVERSITY OF MICHIGAN 311 901 5 047353696 3 9015 04735 3696