Evaluating the effect of sample size changes on scoring system performance using bootstraps and random samples - Section on Survey Research Methods - Brief Article

Currently, the U. S. Internal Revenue Service calculates a scoring formula for each return and uses it as one criterion to determine which returns to audit. Periodically, IRS updates this formula from a stratified random audit sample. In 1988, such an audit sample was selected. The sample was used to derive a new scoring formula. Every return was then scored and sorted by descending scores. The top x percent, say 5 percent, of the returns then determines the "hit rate," the percentage of that 5 percent of the returns that had noncompliance exceeding a selected threshold.

To design a new audit sample, we wanted to know how a change in sample size of 20 percent or more would affect the "hit rate." Since the IRS scoring system is confidential, we simulated it using discriminant analysis. Thus, the "hit rate" is a nonlinear estimator of unknown characteristics. To analyze the effect of sample size change on such a scoring system, we selected random subsamples and balanced bootstrap subsamples and calculated average subsample estimates of the "hit rate" and their variances. This paper presents the design and the results of this analysis.

COPYRIGHT 2002 U.S. Government Printing Office
COPYRIGHT 2004 Gale Group