Division of Research Graduate School of Business Administration The University of Michigan August 1977 SEQUENTIAL SAMPLING METHODS FOR TESTING A COMPLIANCE OBJECTIVE Working Paper No. 150 by R.W. Andrews and J.T. Godfrey The University of Michigan copyright 1977 by The University of Michigan FOR DISCUSSION PURPOSES ONLY None of this material is to be quoted or reproduced without the express permission of the Division of Research

I. The Problem In order for an auditor to execute the second standard of field work properly there must be an evaluation of a client's internal control procedures. For this paper we assume that the internal control procedures to be relied upon have already been determined and what remains is to test for a client's compliance to these controls. For the purpose of this paper, we further assume there is only one attribute upon which the auditor wishes to rely. Thus we are conducting a test of compliance on that one attribute. The problem we are addressing has a special nature in that the internal control procedure we wish to test for is carried out at more than one site. Because the sites are geographically separated, a reasonable question that arises is whether or not an auditor must test all sites in order to come to a conclusion about compliance with the internal control procedure. From our experiences we conclude that this is a common question and to resolve it some auditors still inspect procedures at all the sites, while others subjectively choose a few (maybe two or three) at which to evaluate the internal control procedures. In the latter case, they usually change the sites they inspect from year to year. For example, we can paraphrase a typical statement that we have heard from various auditors as follows, "'If the same internal American Institute of Certified Public Accountants, Statement on Auditing Standards No. 1 (New York: AICPA, 1973), p. 5.

- 2 - control procedures are used at different sites, it seems that if we have inspected a couple sites and found them complying with their procedures we should be able to make a statistical statement about all sites as data is obtained sequentially from site to site." In order to be able to make a statistical statement about all sites, we must develop a model which accounts for the relationship between the error rates at different sites. That is, we expect different sites to have different error rates and our model must reflect this expectation. We do this in Section III below as well as develop a method of stating a conclusion about the upper precision limit achieved. The use of the phrase "upper precision limit" is analogous to the conventional usage, except that we will use it to refer to multiple sites rather than one. We are only considering the evaluation of internal control, i.e., compliance testing, and are not considering the substantive test. In order to crystallize this point, the reader may consider the proposed procedures as being intended for an interim period in which the substantive test will be accomplished at year-end, and we want an interim evaluation of the upper precision limit for the error rate associated with the attribute on which we want to rely. A situation in which there is only one attribute on which to rely is not unrealistic. If the internal control procedures

- 3 - are not stringent the single attribute might simply be the difference between the audit and book dollar value. Since we are at interim, however, we will not be considering the substantive error but only whether or not an error exists. In Section II we introduce a specific example problem. In Section III we formulate and apply a method of solution for the problem; in Section IV we test the validity of our solution method through a simulation study. Section V contains a summary and suggestions for further research. II. Example Problem Consider a situation in which an auditor is responsible for performing a compliance test of the internal control procedures of a manufacturing company which has six different sites. Cash disbursements are being audited and the auditor has decided that the only attribute on which to rely is the agreement or disagreement of the voucher register with the listing of cash disbursements purported to be correct. It is an interim period and our immediate goal is to make a statement about internal controls, since substantive tests will be executed at year end. The information contained in Table 1 is available to us. In Table 1, N. is the number of cash disbursements to date this year at the given site j, and p* is a subjective evaluation of the expected error rate at the site. For example, p. might simply be the error rate which was observed I

- 4 - Table 1 Information Available for Evaluation of Compliance with Internal Control Procedure Site N. p A 1565.04 B 2313.03 C 2216.03 D 1700.03 E 322.02 F 312.02 N. = number of cash disbursements to date ] this year at site j p. = subjective evaluation of error rate at P site j the last time site j was audited. However, given additional information subsequent to the last audit, pj might be the result of subjectively changing the error rate last observed. For example, at Site B assume that the actual error rate observed last year was only 1 percent. However, the volume of inventory at Site B has almost doubled with no additional manpower resources added, therefore, the auditor expects the error rate to increase to approximately 3 percent as given in the table. It follows then that the information content of p. is the result of a history of experience which an auditor will accumulate over a number of periods of auditing the same client. This experience factor is present in most auditing situations and we include it as an important input to the solution approach we present in Section III.

- 5 - II-a. The statistical test Since the audit objective of the present test is to be able to make a statement about the reliability of the internal control procedure we are primarily concerned with two quantities, the reliability (R) and the upper precision limit (u). When we are finished with this compliance test we want to be able to state that with a reliability of.95 we are confident that no site has an error rate greater than u. A general description of the strategy of our compliance test is as follows: After we have audited two sites, we will be able to make a statistical statement about the upper precision limit for all six sites at a specific reliability level. If this stated upper precision limit and reliability are within the ranges we wish to accept, then we can stop without auditing the remaining four sites. Otherwise we will proceed sequentially to the third site and calculate a new upper precision limit. It might be necessary to audit all six sites, especially if the error rates are higher than expected and above some prescribed level. On the other hand, if the upper precision limit remains higher than expected after the inspection of three or four sites, it may be realistic to conclude that the internal control procedure is not accomplishing its job and therefore should be relied on less, rather than expending the cost and effort to inspect the remaining sites.

- 6 - II-b. Selection of sequence of site inspections As will be seen in Section III, the validity of our solution model does not depend on the order in which sites are inspected; it only depends on knowing N. and assessing p*. Thus, a decision about the order in which sites should be inspected is not part of our model. This allows an auditor to make that decision based upon other important factors not included in our model. For example, an auditor may wish to consider the total dollar value of transactions at each site, which would reflect an auditor's ever-present concern about materiality, or the time span since a site was last inspected and/or the magnitude of the expected error rate. If a site has not been inspected for two or more periods and/or is expected to have a higher than normal error rate, then it should be a prime candidate for inspection. These other factors have intuitive appeal and may well lead to the development of a model for optimal sequencing of site inspection. Indeed, this is a potential area for expansion of the model we present here. In this paper, however, the order of site inspection has no impact on the statistical validity of our model. For convenience, in our example problem we select the order of inspection to be the order of sites found in Table 1. II-c. Determination of sample size Since our overall objective is to state an upper precision limit at a specified reliability for all sites we must

- 7 - be able to estimate the error rate at each of the sites already audited. Therefore, an objective at each site inspected is estimation, and we must require a sample large enough to accomplish this. The sample size determination at each site audited will be based on the following four factors: Nj, the total number of transactions at each site; pt, the subjectively evaluated expected error rate; R, the required reliability, and u, a desired upper precision limit. The setting of a desired upper precision limit should be understood for its primary function, which is the establishment of a reasonable sample size. Once the data has been observed the auditor will have to decide whether or not the upper precision which is achieved is acceptable. The proposed random sample at each site is without replacement so we will employ the hypergeometric distribution. If R = reliability, N. = the total number of transactions, pj = the subjectively evaluated expected error rate, and u = desired upper precision limit, then a reasonable sample size will be the smallest value of n. such that uN. N.-uN penj iui (\nj-uN) < - R. (2.1) i=O Nj nj If N. is large compared to the sample size we can approximate the hypergeometric distribution with the normal distribution. That is, in a random sample without replacement of size n. 3

-8 - from a finite population as described above, the random variable Yj, the number of observed errors at site j, will have an approximate normal distribution with the following mean and variance: Mean = n.u N.-n. Variance = n.u(l-u) N (2.2) J Our purpose is to find a sample size which would yield an observed error rate less than p* only (l-R) of the time if the population error rate were u. This means that, if the population error rate is u or larger, we would most likely (R of the time) observe an error rate pj or larger. This stipulation can be stated using our normal approximation as follows. Find the smallest n. such that J P[Y. < np] = 1 - R (2.3) or Y.-n.u n.p*-n.u P 3 _ < 3 J 3 = 1 - R. (2.4) Y/ N.-n. N.-n. |u 3I/nj u(l-u) u(l-u) Therefore, n.p *-n.u = 3 31 - Z / - = -Zl1-R (2.5) N.-n. n.u(l-u) I J3 N.-l J

- 9 - where Z is the value of the standard normal random variable with a probability of a in one tail. Solving (2.5) for n. we have Z2 u(l-u)N. n. =2 2 (2.6) Zl RU(l-u) + (Nj-l) (pj-u) For the case we are considering with u =.08 and R =.95 we find the sample sizes for the six sites which are set forth in Table 2. Table 2 Sample Sizes for Each Site Site n. A 116 B 78 C 77 D 77 E 48 F 47 In the next section we present a model and procedure for evaluating the sample data. III. Formulation of a Solution Procedure In this section we will first describe a model that is assumed for the problem described above; then we will derive estimation procedures used to evaluate this model. This methodology will yield a sequential process which will give an achieved upper precision limit after each site in

- 10 - spection beginning with the second site inspected. We require that at least two sites always be inspected so that we will have some additional evidence as to the variability of the error rate between sites. III-a. Model formulation Let p be the random variable that is the proportion of errors generated by the accounting procedures being employed at all sites. We assume a Beta distribution on p with unknown parameters (a,b). We write the density of p to be f(p) r(a + b) a-I b-l O p' f(p) -' D (l-p) 0 < 1 r(a)r(b) (3.1) a, b > 0. This gives us a rich family of densities for possible distributions on p. It should be noted that the actual proportion of errors, pj, for each of the sites will be a realization of the random variable p. Consistent with an earlier definition, we let p; (j = 1, -*, k) be the subjectively evaluated expected error rate that has been assessed for each site. We assume that p* is related to p through a function, i.e., p* = h(p). Since p is a random variable and p* is a function of p, then p* will be a random variable. We do not assume a specific form for h(-); however, as part of the model we assume that h(-) is of a form such that

- 11 - (i) E[p] < E[h(-)] ab (ii) Var[h(-)] = Var[p] =ab (a+b) (a+b+l) The reason for these assumptions is that we will use the p*'s in the initial estimate of the mean and variance of p. Assumption (i) implies that using the pt's to estimate the mean of p will result in a conservative estimate, since we should get a positively biased estimate and this will lead to a positively biased upper precision limit. Assumption (ii) allows us to use the variance of the pt's as a basis for estimating the variance of p. However, as we shall see later, the sample data will be allowed to augment the p*'s in making the variance estimate as well as the estimate of the mean. After we have sequentially inspected 9 sites, we will have a problem structure as given in (3.2). The pj's and pj's form the data set which will be used in our estimation procedures. Site 1 Site 2 Site z Site ~ + 1 Site k Pi P P P rP+1P (3.2) P?2 P The Pl, ''', pk are unknown realizations from the distribution given in (3.1). The pl,, pk are observed

- 12 - Y. realizations from the distribution p* = h(p) and p. = - j n. (j = 1 *,, ), the sample proportion of errors at site j for the sites inspected. Given our data set in (3.2), we want to develop estimates of a and b, the parameters in the distribution of p (3.1). The following discussion will lead to acceptable estimators. Our strategy for estimating the parameters a and b will be to use our data set in (3.2) to calculate estimates ^A2 (p,a ) of the mean and variance of (3.1). Then since we know that a = a+b (3.3) a+b and 2 ab. = ' - -2, (3.4) (a+b) (a+b+l) ^2 we can equate these to i and a and solve for a and b yielding ^2 ^ ^2 a= (1-1 - ) (3.5) ^2 and. 2 A2 ) b = (-)2 (3.6) 52 In developing estimators of p and a2 from our data set we have attempted to model typical auditing logic within our statistical estimation procedure. That is, in accomplishing our objective of calculating an upper precision limit u* for

- 1J - all k sites after z. < k sites have been inspected, we would rather overstate u* than understate it at our specified reli-_abiity level..-Another auditing logic characteristic that we desire in our estimation procedure is that, for a given set of sites inspected, the larger the observed error rates, the larger will be u*. For example, if sites 1, 2 and 3 are inspected with observed error rates p1 =.03, P2 =.04, P3 =.03, respectively, we want u* to be larger than it would be if the results would have been p1 =.02, P2 =.01, P3 =.03. These objectives are incorporated into our esti2 mation procedures for p and 2. As an estimator of p we use Pl p + P2 + ''' + P + * + + P kp1 = 1 ------- ~ ----(3.7) = k ' This estimator uses the latest data available, the pj's (j = 1,, 9) for the first 2 sites inspected (in place of the first a p*'s). Since it is a blend of pj's and pj's it will tend, on the average, to be positively biased (as mentioned earlier), because E[p*] > E[p] and E[p] = E[p]. An upward biased p may lead to a larger u*. However, as more sites are inspected, p will have less bias as an estimator of p because we are using more pj's in place of the pj's. This is acceptable from an auditing logic point of view as we are using more currently available data (the pj's) and therefore p should become a better estimate of p. When we were using more pj's relative to the pj's we preferred to be conservatively biased (E[p*] > E[p]) rather than otherwise.

- 14 - A variance estimator (a ) is not as straightforward to develop as the mean estimator (). Since Var[p*] = Var[p] = 2 o, one approach is to use k I (p-p*)2 2 i=l ks (3.8) Te wt k2 ^2 2 2 Then with al = sp as an estimator of a, we could use it in 1 p* (3.5) and (3.6) along with P to calculate estimators a and i. 2 However, s p does not use any data from the Z site inspections and, therefore, we elect not to use it and to improve on it using the complete data set. The pj's should be incorA2 porated into a since they represent the most current data regarding the true error rates (pj) being generated. This leads to another approach which is to consider Var[p]. Var[p] = E{Var[p|p]} + Var{E[plp]} = E{P(p N - 1} + Var[p]. (3.9) n N 1 So, Var[p] = Var[p] - E{P(1-p) _ N- } n N- 1 Therefore, another possible estimator of Var[p] would be n,2 2 1 ^-1 t P.( i N. ~ n-p A2 ^ S 1 x- -3 _ (3.10) [2 P n. N. 1 j= - n J wee P S j. g2 p PI2

- 15 - n2 We also object to using 02 as an estimate of Var[p]. The main reason is that, because of the relatively low values of pj with which we are working, it is not unusual for our ^2 sample data to yield a a2 < 0. This, of course, is not a A2 usable result. Two other objections to 2 are (1) that it does not make use of any of the prior knowledge embodied in the p*'s and (2) for small L there may be too few pj's available for an adequate estimate. ^2 ^2 Besides the short-comings of 01 and a2 discussed above, we also desire that our estimator of Var[p] help us to achieve our auditing objectives mentioned earlier. In brief, these objectives were not to understate an achieved upper precision limit (u*) and that our calculation of u* be consistent with the error rates observed at sites inspected. With all of our objectives in mind, we propose the following estimator of Var[p] (p -p*)2 + x(p* -p)2 ^2 A B J 3 = Ak (3.11) where the set A consists of all sites inspected where the observed pj > pj and fP j - P*| > Ij* - p*| and set B consists of all sites not included in A. An observed pj will enter the calculation only if (1) pj is larger than its corresponding pj

- 16 - and (2) the distance that pj is from p* is larger than the distance the corresponding p* is from p*. A2 We elect to keep p* constant in our calculation of a3 (i.e., A2 not recalculated using the pj's) so that the value of a3 will always be at least as large as the estimator based only on the p*'s (a2). Then since we assume Var[p] = Var[p*] and a2 t 2 1l is an estimate of Var[p*], we are confident that a3 will be an upward biased (conservative) estimated of Var[p]. This should have a favorable impact on our auditing objectives as well as meet our statistical requirements. Now that we have determined our estimators p [in ^2 (3.7)]and a3 [in (3.11)]we can use them to calculate estimates of a and b, the parameters of the Beta distribution which is generating the error rates at the k sites. With a and S we have an estimate of the distribution of p and, therefore, can make a statement concerning the probability that any site will have an error rate above a certain level. After inspecting i sites, we can state with reliability R that u* is the achieved upper precision limit for all k sites if u* = min{Prob[max p. u] 2 RI. (3.12) u J This expression states that the probability that the largest pj will be less than or equal to u* is greater than or equal to the specified reliability. The probability calculation

- 17 - uses the Beta distribution with the estimated values of a and b. In order to evaluate this procedure, we need to use an approximation to an incomplete Beta integral. Assuming independence of the realization of the pj's we have Prob[max pj u] = {Prob[p ' u]}k { r(a + ) pa-l( p)b-ldpk (3.13) o r(a) + r(a) Setting this probability equal to R, we are left with the numerical problem of finding the value of u* such that, u* -1 7f r( (1 - p) pa- dp = R (3.14) 0 r(a) + r(b) 2 An algorithm has been given which solves this problem and which we use in our numerical example. The solution, u*, is the achieved upper precision for all sites. We next consider calculating u* for the example problem introduced in Section II. III-b. Numerical results of example problem In order to demonstrate our suggested procedure, we will estimate the upper precision limit for all sites after K.L. Majumder and G.P. Bhattacharjee, "Inverse of the Incomplete Beta Function Ratio," Applied Statistics 22, No. 3 (1973): 411-14.

- 18 - the first two sites have been inspected (k = 2). Recall that there are six sites in total (k = 6) with population sizes and p' s given in Table 1 and sample sizes given in Table 2. Table 3 below gives a calculation of u* after the inspection of two sites and for seven possible error outcomes at each site (number of errors Y. = 0, 1, *-*, 6). Table 3 Upper Precision Limit for All Six Sites after First Two Sites Inspected (Reliability =.95) Site B (nB = 78) 0 1 2 3 4 5 6 p. 0.0128.0256.0385.0513.0641.0769 0 0.0372.0388.0406.0462.0606.0792.1000 1.0086.0383.0400.0418.0474.0616.0800.1007 2.0172.0394.0412.0431.0486.0627.0808.1011 3.0259.0407.0425.0444.0498.0637.0818.1020 4.0345.0419.0437.0457.0511.0649.0827.1026 5.0431.0462.0480.0499.0550.0680.0852.1046 6.0517.0576.0592.0608.0651.0764.0921.1103 Site A (nA = 116) Table 3 may be read as follows. If at Site A we observe YA = 2 errors and at Site B, YB = 3 errors, we calculate u* =.0486 as our achieved upper precision limit. That is, we believe that with probability.95 none of the six sites has an error rate (pj) larger than.0486. This limit is calculated, of course, from a specific Beta distribution with

- 19 - parameters a and b whose values were dependent on our estiA A2 mators p and a3. Notice that u* increases monotonically as we read down and to the right across Table 3. This reflects one of the auditing objectives stated earlier in Section III-a. Our other primary auditing objective (that u* not be understated) cannot be demonstrated in Table 3, but we will comment on it in the report of our simulation runs in Section IV. To further illustrate the monotonic property, we calculated the u*'s after three sites had been inspected (Sites A, B, and C), given that two possible occurrences had been observed at each of Sites A and B. That is, we assumed that YA = 2 and YB = 5 at Sites A and B and then calculated u* for the possible outcomes of YC = 0, 1, *', 6. We then repeated the calculations assuming YA = 5 and YB = 5. The results are given in Tables 4 and 5. Table 4 Upper Precision Limit for All Six Sites after First Three Sites Inspected and YA 2, B 5 (Reliability =.95) Site C (nC = 77) Yc 0 1 2 3 4 5 6 p 0.0130.0260.0390.0519.0649.0779 u*.0782.0791.0805.0839.0933.1069.1237

- 20 - Table 5 Upper Precision Limit for All Six Sites after First Three Sites Inspected and YA = 5, YB = 5 (Reliability =.95) Site C (nC = 77) Yc 0 1 2 3 4 5 6 PC I 00.0130.0260.0390.0519.0649.0779 u*.0821.0834.0848.0882.0973.1105.1269 When only two sites were inspected with YA = 2 and YB = 5, u* was.0808 (see Table 3). Then in Table 4, we see that with zero errors observed at site C, u* decreases to.0782. This is not surprising since our new data from Site C is an improvement over the error rates observed at Sites A and B. However, we see that u* increases monotonically as the error rate increases at Site C and for p =.0390, u* =.0839, which is larger than it was at the end of inspection of Site B. In the other case when only two sites were inspected with YA = 5 and Y = 5, u* was.0852 (see Table 3). Again, observing pC = 0 reduces u* to.0821, but then u* increases as more errors are observed, and at pC =.0390, u* =.0882, which is greater than it was at the end of inspection of Site B.

- 21 - Thus, our calculation of u* is responsive to variation in error rates observed between sites inspected. Also, u* is consistent for a given site since it increases monotonically as the observed error rate (pj) increases. In Tables 6 and 7 below, we compare our results for the example problem with the calculation of the upper precision limit by traditional methods. In Table 6 we consider the case of inspecting Sites A and B. Notice in Table 6 that upper precision limits for Sites A and B are calculated independently of each other by the traditional method. Then the decision to inspect more sites would depend on, among other things, a subjective evaluation of the error rates at the remaining sites. Our proposed u* is an upper precision limit for all sites, whether or not inspected. There may be other criteria that will influence an auditor's decision about whether or not to inspect more sites. However, in u* he does have a statistically derived error rate which applies to all sites and which can be used to help make his decision. In Table 7 consider the case where Sites A and B have been inspected with two different outcomes for each and then consider seven possible outcomes for Site C. In the top part of Table 7 we have error rates of PA =.0172 and B =.0641 and seven different error rates for Site C. Again, all of the traditional upper precision limits are calculated independently of each other while each of our proposed upper

- 22 - Table 6 Upper Precision Limit (U.P.L.) Comparisons Two Sites Inspected (Reliability =.95) Sites A B C D E F Sample Size 116 78 --- --- -- ---- Observed Errors 2 5 -—.... |.. --- Error Rate.0172.0641.... --- - ------ Traditional U.P.L.*.0600.1400. --- --- --- Proposed U.P.L..0808.0808.0808.0808.0808.0808 Sites A B C D E F Sample Size 116/ 78 --- --- --- ---- Observed Errors 5 5 -_ --- _ _ —.-.. Error Rate.0431.0641 ---- --- - --- Traditional U.P.L.*.0900.1400 ---- --- -- Proposed U.P.L..0852.0852.0852.0852.0852.0852 *Sampling for Attributes (Supplementary Section), AICPA, 1967, page S-21, Table 2-B. Upper precision limits for n = 80 and 120 are used since the source table does not include n = 78 and 116.

- 23 - Table 7 Upper Precision Limit Comparisons Three Sites Inspected (Reliability =.95) Sites. I. A B C Sample Size 116 78 77 77 77 77 77 77 77 Observed Errors 2 5 0 1 2 3 4 5 6 Error Rate.0172.0641.0000.0130.0260.0390.0519.0649.0779 Traditional U.P.L.*.0600.1400.0400.0600.0800.1000.1200.1400.1500** Proposed U.P.L. (for all sites) --- ---.0782.0791.0805.0839.0933.1069.1237 Sites A B C Sample Size 116 78 77 77 77 77 77 77 77 Observed Errors 5 5 0 1 2 3 4 5 6 Error Rate.0431.0641.0000.0130.0260.0390.0519.0649.0779 Traditional U.P.L.*.0900.1400.0400.0600.0800.1000.1200.1400.1500** Proposed U.P.L. (for all sites) --- ----.0821.0834.0848.0882.0973.1105.1269 *Sampling for Attributes (Supplementary Section), AICPA, S-21, Table 2-B. Upper precision limits for n = 80 and since source table does not include n = 77, 78 and 116. ** Interpolated since no table value given for 6 errors. 1967, page 120 are used

- 24 - precision limits is dependent on the outcomes at all three sites and pertain to all six sites. In the next section we use a simulation study to test the validity of our approach. IV. Simulation Study The main purpose of this simulation study will be to demonstrate the validity of the procedures described in the last section. In the last section we reported procedures for calculating an upper precision limit u* at a specified reliability R. Using simulation we will show that u* actually has that reliability. Because of the complex nature of the estimation procedure an analytic proof is intractable; therefore stochastic simulation methods will be employed. IV-a. Simulation design We have designed a simulation experiment which performs 600 replications of the specific problem discussed in Section III. In order to investigate this specific problem we have fixed: (1) the number of sites at six, (2) the population sizes and the subjectively evaluated error rates as given in Table 1, and (3) the sample sizes as given in Table 2. We could use other values for these quantities; however a favorable conclusion about the reliability for this particular problem will give credence to the statistical validity of the procedure.

- 25 - Each replication can be considered as a compliance test for a company with six sites. It still holds that there is only one attribute on which we have decided to rely. Since the procedures for arriving at an upper precision limit, as given in the last section, will be applied to each application the reliability levels will be stated as a percentage of the 600 outcomes. The first random element of each replication is a Beta distribution which generates the actual error rates at each site. We decided to set the mean and variance of the generating distribution at the mean and variance of the six p*'s. Since the mean and variance of the p*'s are.028333333 and.000047222 respectively, the resulting parameters of the Beta distribution are a = 16.49 and b = 565.51. Therefore for this simulation study we are generating the pj's from a distribution for which E[p] is equal to the sample mean of the p*'s and the Var[p] is set at the sample variance of the p*'s. In our model we assumed that Var[p] = Var[p*]; therefore, for simulation purposes it is only reasonable that our initial study should set the generating variance at the sample variance of the p*'s. If we wanted the simulation study to conform explicitly with our model assumptions, then we should set the mean of the generating distribution at a value smaller than the mean of the p*'s. Since the difference between E[p] and E[p*] was not specified in the model, we set the generating mean

- 26 - equal to the average of the p*'s. A sensitivity analysis, not reported here, which investigates different parameter settings for the generating distribution is underway. The second random element of each replication is a sample taken without replacement at each site. Based on these samples and using the procedures given in Section III, an upper precision limit is calculated after the 2nd, 3rd, 4th, 5th and 6th sites. Since we know the actual error rates we can determine the percentage of times that our stated upper precision limit exceeded the error rates at all of the sites. These percentages are an estimate of the reliability of our procedure. IV-b. Simulation results Throughout this study we used R =.95. Table 8 gives the proportion of times that none of the six actual error rates exceeded our u* for 9 = 2, 3, 4, 5, 6, where 9 is the number of the sites inspected. Table 8 Simulation Study Achieved Reliability Proportion of Times None of Actual Number of Error Rates Sites Inspected Exceeded u* 2.94 3.94 4.945 5.962 6.978

- 27 - These values indicate that our procedure has a reliability of approximately.95. Notice the increase in reliability as the number of sites inspected increases. This is due to the fact that the estimate of the variance of the generating distribution can only increase as Z, the number of sites inspected, increases. However, the importance of these results is that u* will be reported at approximately a.95 reliability. This can be further emphasized by considering the following. For each of the 600 replications a u* was reported after the 2nd site was inspected. Of the 600 u*'s only 36 (.06) failed to exceed the error rates of all six sites. With this data we would accept the hypothesis that the reliability is.95. V. Conclusions and Further Research We have modeled the situation in which the same internal control processes are being applied to several different sites. On the basis of our model we have developed a method of stating an upper precision limit (u*) for all the sites at a specified reliability (R). The interpretation of this (u*,R) pair should be that we are R confident that no site has an error rate which exceeds u*. The estimation procedure was developed on the basis of logical auditing properties and was not developed with any particular statistical property in mind. However, we were able to show the statistical validity of our reliability statement by using simulation. The

- 28 - simulation study indicated that when we set R =.95, our procedure gave an upper precision limit that exceeded all error rates approximately.95 of the time. We have developed a procedure for stating an upper precision limit for a single attribute when there are multiple sites. This same procedure could be used independently on more than one attribute; however, we believe that there should be more research in the multiple attributes case. In fact, very little work has been done for the multiple attributes case when there is only one site.